Job Scheduling Optimization of High Performance Computing in Biological Gene Sequencing Based on Workload Analysis

Mining job scheduling features based on extraction and analysis of workload trace in high performance computing clusters can be used to optimize scheduling strategy and enhance system performance. Based on detailed analysis of workload trace from a gene sequencing high performance computing system, this paper proposes a multi-queue backfilling scheduling algorithm, which is based on traditional backfilling scheduling. While optimizing for memory resource demands, this algorithm provides queue level load balancing to deal with the innate load imbalance characteristics of high performance systems. Experimental results based on practical gene sequencing workload trace clearly demonstrate that compared with traditional scheduling algorithms, the algorithm proposed in this paper is a good strategy to reduce the job waiting time and improve resource utilization.