Interference-aware Multiplexing for Deep Learning in GPU Clusters: A Middleware Approach
Wenyan Chen, Zizhao Mo, Huanle Xu, Kejiang Ye, Chengzhong Xu
Abstract
A common strategy for improving efficiency in training deep learning entails multiplexing tasks on a single GPU. To mitigate the interference caused by multiplexing, existing approaches primarily employ kernel-level solutions to regulate GPU kernel execution, or harness hardware-level techniques to explicitly restrict GPU streaming multiprocessors and memory. Nevertheless, none of them perform satisfactorily in optimizing the completion time of tasks.
Topics & Concepts
Computer scienceMultiplexingKernel (algebra)Interference (communication)Middleware (distributed applications)Deep learningComputer architectureParallel computingDistributed computingArtificial intelligenceComputer networkTelecommunicationsMathematicsChannel (broadcasting)CombinatoricsAdvanced Neural Network ApplicationsStochastic Gradient Optimization TechniquesParallel Computing and Optimization Techniques