DRAS-CQSim: A reinforcement learning based framework for HPC cluster scheduling
Yuping Fan, Zhiling Lan
Abstract
For decades, system administrators have been striving to design and tune cluster scheduling policies to improve the performance of high performance computing (HPC) systems. However, the increasingly complex HPC systems combined with highly diverse workloads make such manual process challenging, time-consuming, and error-prone. We present a reinforcement learning based HPC scheduling framework named DRAS-CQSim to automatically learn optimal scheduling policy. DRAS-CQSim encapsulates simulation environments, agents, hyperparameter tuning options, and different reinforcement learning algorithms, which allows the system administrators to quickly obtain customized scheduling policies.
Topics & Concepts
Reinforcement learningComputer scienceScheduling (production processes)Distributed computingSupercomputerHyperparameterCluster (spacecraft)Computer clusterArtificial intelligenceParallel computingOperating systemMathematical optimizationMathematicsCloud Computing and Resource ManagementReinforcement Learning in RoboticsDistributed and Parallel Computing Systems