CoDL
Fucheng Jia, Deyu Zhang, Ting Cao, Shiqi Jiang, Yunxin Liu, Ju Ren, Yaoxue Zhang
Abstract
Concurrent inference execution on heterogeneous processors is critical to improve the performance of increasingly heavy deep learning (DL) models. However, available inference frameworks can only use one processor at a time, or hardly achieve speedup by concurrent execution compared to using one processor. This is due to the challenges to 1) reduce data sharing overhead, and 2) properly partition each operator between processors.
Topics & Concepts
Computer scienceSpeedupPartition (number theory)Parallel computingInferenceOverhead (engineering)Operator (biology)Distributed computingProgramming languageArtificial intelligenceChemistryBiochemistryCombinatoricsMathematicsTranscription factorRepressorGeneAdvanced Neural Network ApplicationsMachine Learning and AlgorithmsAdversarial Robustness in Machine Learning