Characterizing Power Management Opportunities for LLMs in the Cloud
Pratyush Patel, Esha Choukse, Chaojie Zhang, Íñigo Goiri, Brijesh Warrier, Nithish Mahalingam, Ricardo Bianchini
Abstract
Recent innovation in large language models (LLMs), and their myriad use cases have rapidly driven up the compute demand for datacenter GPUs. Several cloud providers and other enterprises plan to substantially grow their datacenter capacity to support these new workloads. A key bottleneck resource in datacenters is power, which LLMs are quickly saturating due to their rapidly increasing model sizes.
Topics & Concepts
BottleneckCloud computingKey (lock)Computer scienceResource (disambiguation)Plan (archaeology)Power (physics)Resource management (computing)Distributed computingComputer securityOperating systemComputer networkEmbedded systemGeographyArchaeologyPhysicsQuantum mechanicsCloud Computing and Resource ManagementSoftware System Performance and ReliabilityParallel Computing and Optimization Techniques