PipePar: Enabling fast DNN pipeline parallel training in heterogeneous GPU clusters
Jinghui Zhang, Geng Niu, Qiangsheng Dai, Haorui Li, Zhihua Wu, Zhihua Wu, Fang Dong, Zhiang Wu, Zhiang Wu
Topics & Concepts
Computer sciencePipeline (software)Parallel computingSymmetric multiprocessor systemPruningCUDAGPU clusterHeterogeneous networkSpeedupArtificial neural networkDistributed computingComputer engineeringArtificial intelligenceBiologyTelecommunicationsAgronomyWireless networkProgramming languageWirelessAdvanced Neural Network ApplicationsDomain Adaptation and Few-Shot LearningMachine Learning and ELM