Providing load flexibility by reshaping power profiles of large language model workloads

Abstract

The emergence of large language models (LLM) has driven a significant increase of AI workload in data center power demand. Renewable-powered solutions to decarbonizing LLM workload and reducing electricity costs are faced with the combined volatility of stochastic user requests and renewable energy. The key to removing the barriers in sustainable AI development lies in the adjustable capability of LLM power profiles. Therefore, this paper focuses on exploring the potential load flexibility of LLM workload and proposes a coordinated scheduling framework, notably, without computing performance degradation. Driven by the existence of the energy-optimal core frequency for graphics processing units (GPU), the energy-performance decoupling phenomenon is discovered and proved, where collaborative scaling in GPU quantity and frequency can change power but not computing performance. Motivated by this, the framework slows down the fine-tuning cluster and utilizes idle GPU resources from the inference cluster to maintain the computing performance of fine-tuning tasks. Consequently, the power consumption of the total cluster is reduced, which provides a fresh source of load flexibility. Furthermore, the framework employs dynamic frequency scaling to more flexibly modify the power profile of the expanded fine-tuning cluster. The computing performance is particularly guaranteed through temporal coupling constraints. In a simulated study supported by real-world data, the results prove a 6.8% power-saving ability and 11.3% cost-saving gains on average.

Topics & Concepts