VELTAIR: towards high-performance multi-tenant deep learning services via adaptive compilation and scheduling
Zihan Liu, Jingwen Leng, Zhihui Zhang, Quan Chen, Chao Li, Minyi Guo
Abstract
Deep learning (DL) models have achieved great success in many application domains. As such, many industrial companies such as Google and Facebook have acknowledged the importance of multi-tenant DL services. Although the multi-tenant service has been studied in conventional workloads, it is not been deeply studied on deep learning service, especially on general-purpose hardware.
Topics & Concepts
Deep learningComputer scienceScheduling (production processes)Artificial intelligenceService (business)EngineeringBusinessOperations managementMarketingDistributed and Parallel Computing SystemsCloud Computing and Resource ManagementAge of Information Optimization