Optimal Kernel Orchestration for Tensor Programs with Korch
Muyan Hu, Ashwin Venkatram, S. Biswas, Balamurugan Marimuthu, Bohan Hou, Gabriele Oliaro, Haojie Wang, Liyan Zheng, Xupeng Miao, Jidong Zhai, Zhihao Jia
Abstract
Kernel orchestration is the task of mapping the computation defined in different operators of a deep neural network (DNN) to the execution of GPU kernels on modern hardware platforms. Prior approaches optimize kernel orchestration by greedily applying operator fusion, which fuses the computation of multiple operators into a single kernel, and miss a variety of optimization opportunities in kernel orchestration.
Topics & Concepts
OrchestrationComputer scienceKernel (algebra)Tensor (intrinsic definition)MathematicsDiscrete mathematicsPure mathematicsArtMusicalVisual artsTensor decomposition and applicationsModel Reduction and Neural NetworksNumerical Methods and Algorithms