Multi-resource interleaving for deep learning training
Yihao Zhao, Yuanqiang Liu, Yanghua Peng, Yibo Zhu, Xuanzhe Liu, Xin Jin
Abstract
Training Deep Learning (DL) model requires multiple resource types, including CPUs, GPUs, storage IO, and network IO. Advancements in DL have produced a wide spectrum of models that have diverse usage patterns on different resource types. Existing DL schedulers focus on only GPU allocation, while missing the opportunity of packing jobs along multiple resource types.
Topics & Concepts
InterleavingComputer scienceFocus (optics)Resource (disambiguation)Deep learningResource allocationArtificial intelligenceDistributed computingComputer architectureMachine learningParallel computingComputer networkOperating systemOpticsPhysicsAdvanced Neural Network ApplicationsCloud Computing and Resource ManagementParallel Computing and Optimization Techniques