Heterogeneous 3-D Integration of Multitier Compute-in-Memory Accelerators: An Electrical-Thermal Co-Design

Abstract

Emerging nonvolatile memory (eNVM)-based compute-in-memory (CIM) accelerators have been proven in silicon for machine learning at the macrolevel. To fully unleash the system-level benefits, the heterogeneous 3-D integration (H3D) using through-silicon via (TSV) is a promising approach, to: 1) address the challenges of area-hungry peripheries in CIM accelerators; 2) solve the 2-D scaling challenges of eNVM; and 3) stack enormous amount of embedded memories that are required in state-of-the-art deep neural network models. This article presents an electrical-thermal co-design of multitier CIM accelerators, based on SRAM and/or eNVM, with hybrid technology nodes for logic and memory tiers. We benchmark the CIM accelerators on 8-bit ResNet-34 for ImageNet recognition, with layer-by-layer and pipelined schemes, respectively. By sweeping TSV diameter from <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$30~\mu \text{m}$ </tex-math></inline-formula> to 100 nm, we investigate the tradeoffs of system performance metrics (TOPS/W, TOPS, and TOPS/mm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> ) and H3D challenges (thermal and IR-drop in power delivery). Finally, we find the sweet spot of TSV diameter for multitier H3D system is 1– <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$3~\mu \text{m}$ </tex-math></inline-formula> , to guarantee balanced area-overhead, performance, and IR-drop in power delivery. The extended benchmark framework is released on GitHub ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><uri>https://github.com/neurosim</uri></i> ) as an open-source tool for the research community.

Topics & Concepts