A 65 nm 1.4-6.7 TOPS/W Adaptive-SNR Sparsity-Aware CIM Core with Load Balancing Support for DL workloads
Mustafa Ali, Indranil Chakraborty, Sakshi Choudhary, Muya Chang, Dong Eun Kim, Arijit Raychowdhury, Kaushik Roy
Abstract
The growing trends of developing domain-specific accelerators for Deep Learning (DL) applications has led to exploration of compute-in-memory (CIM) primitives based on SRAM [1] –[5]. Multiple research chips have demonstrated macro and core-level designs supporting multi-bit Matrix-Vector Multiplication (MVM) and sparsity to increase energy-efficiency and performance. However, CIM designs suffer from the following challenges, as shown in Fig. 1: (1) Difficulty in leveraging both input and weight unstructured sparsity in existing DL accelerators. Note, unstructured sparsity is more amenable during DL model training than structured sparsity. Fig. 1 (top) shows input and weight bit-level sparsity of ResNet20 running a CIFAR10 task and mapped on a $64 \times 64 \mathrm{CIM}$ macro. We observe that activations and weights of each layer experience different bit-level sparsity, also, sparsity levels vary significantly across layers.(2) Mixed-signal CIM macros suffer from noise and variation-based computation errors and signal-to-noise ratio (SNR) degradation. Moreover, the macro errors get accumulated in scaled-up CIM architectures leading to significant model accuracy drop. (3) Sparsity-aware CIM compute units encounter different sparsity; hence they might finish their corresponding MVMs at different times leading to load imbalance.