Heterogeneous Embedded Neural Processing Units Utilizing PCM-Based Analog In-Memory Computing
Irem Boybat, Thomas Boesch, Mario Allegra, M. Baldo, J.J. Bertolini-Agnoletto, Geoffrey W. Burr, Alessandro Buschini, A. Cabrini, E. Calvetti, Carmine Cappetta, Francesco Conti, Elena Ferro, Eleonora Franchi Scarselli, A. Garofalo, Francesca Girardi, Gamze İslamoğlu, Vara Prasad Jonnalagadda, Geethan Karunaratne, Corey Lammie, Manuel Le Gallo, C. Li, Riccardo Massa, Andrea Carlo Ornstein, H. Pang, M. Pasotti, Bipin Rajendran, Andrea Redaelli, I. Sanli, William A. Simon, Abhairaj Singh, Surinder-Pal Singh, Giulio Urlini, Athanasios Vasilopoulos, R. Zurla, Giuseppe Desoli, Abu Sebastian
Abstract
We propose an embedded Neural Processing Unit (NPU) architecture for deep learning inference to address the stringent energy, area, and cost requirements of edge AI. This heterogeneous architecture integrates a variety of digital and analog accelerator nodes to cater to diverse operation types and precision requirements. To achieve high energy efficiency while maintaining substantial non-volatile on-chip weight capacity, we utilize Analog In-Memory Computing (AIMC) tiles based on Phase-Change Memory (PCM) for Matrix-Vector Multiplications (MVMs). Additionally, a digital data path and a programmable software cluster facilitate end-to-end inference across multiple precision levels. The NPU is projected to deliver competitive throughput for transformer Neural Networks (NNs), rivaling high-end System-on-Chips (SoCs) for mobile devices and edge accelerators fabricated at more advanced technology nodes.