Performance, efficiency, and cost analysis of wafer-scale AI accelerators vs. single-chip GPUs
Mihrimah Ozkan, Lily Pompa, Mochammad Iqbal, Yiu Chan, Daniel R. Morales, Zixun Chen, Handing Wang, Lusha Gao, Sebastián González
Abstract
The exponential growth of artificial intelligence (AI) models, now reaching trillions of parameters, has revealed significant limitations in traditional single-chip graphics processing unit (GPU) architectures, particularly in scalability, energy efficiency, and computational throughput. Wafer-scale computing has emerged as a transformative paradigm, integrating multiple chiplets into a single monolithic wafer to deliver unprecedented performance and efficiency. Platforms such as the Cerebras Wafer-Scale Engine (WSE-3), with 4 trillion transistors and 900,000 cores, and Tesla's Dojo, featuring 1.25 trillion transistors and 8,850 cores per training tile, exemplify the potential of wafer-scale AI accelerators to address the demands of large-scale AI workloads. This review provides a comprehensive comparative analysis of wafer-scale AI accelerators and single-chip GPUs, focusing on their relative performance, energy efficiency, and cost effectiveness in high-performance AI applications. Emerging technologies, such as TSMC's chip-on-wafer-on-substrate (CoWoS), which promise to enhance computational density by up to 40 times, are also examined. In addition, this work discusses critical challenges, including fault tolerance, software optimization, and economic feasibility, offering insights into the trade-offs and synergies between these two hardware paradigms. Furthermore, emerging AI hardware trends, including three-dimensional (3D) integration, photonic chips, and advanced semiconductor materials, are also discussed. This review aims to inform the development of scalable and energy-efficient AI computing by evaluating their strengths and limitations. A future outlook outlines key advancements expected over the next 5 to 10 years, shaping the next generation of AI hardware.