Software-Hardware Co-design of Heterogeneous SmartNIC System for Recommendation Models Inference and Training
Anqi Guo, Yuchen Hao, Chunshu Wu, Pouya Haghi, Zhenyu Pan, Min Si, Dingwen Tao, Ang Li, Martin Herbordt, Tong Geng
Abstract
Deep Learning Recommendation Models (DLRMs) are important applications in various domains and have evolved into one of the largest and most important machine learning applications. With their trillions of parameters necessarily exceeding the high bandwidth memory (HBM) capacity of GPUs, ever more massive DLRMs require large-scale multi-node systems for distributed training and inference. However, these all suffer from the all-to-all communication bottleneck, which limits scalability.
Topics & Concepts
BottleneckComputer scienceScalabilityInferenceNode (physics)Bandwidth (computing)SoftwareArtificial intelligenceDistributed computingComputer architectureMachine learningComputer engineeringEmbedded systemComputer networkOperating systemStructural engineeringEngineeringRecommender Systems and TechniquesStochastic Gradient Optimization TechniquesAdvanced Graph Neural Networks