Reducing Energy Bloat in Large Model Training
Jae-Won Chung, Yile Gu, Insu Jang, Luoxi Meng, Nikhil Bansal, Mosharaf Chowdhury
Abstract
Training large AI models on numerous GPUs consumes a massive amount of energy, making power delivery one of the largest limiting factors in building and operating datacenters for AI workloads. However, we observe that not all energy consumed during training directly contributes to end-to-end throughput; a significant portion can be removed without slowing down training. We call this portion energy bloat.
Topics & Concepts
Training (meteorology)Computer scienceEnergy (signal processing)MathematicsMeteorologyPhysicsStatisticsAdvanced Neural Network ApplicationsParallel Computing and Optimization TechniquesCloud Computing and Resource Management