11.3 Metis AIPU: A 12nm 15TOPS/W 209.6TOPS SoC for Cost- and Energy-Efficient Inference at the Edge
Pascal Alexander Hager, Bert Moons, Stefan Cosemans, Ioannis A. Papistas, Bram Rooseleer, Jeroen Van Loon, Roel Uytterhoeven, Florian Zaruba, Spyridoula Koumousi, Miloš Stanisavljević, Stefan Mach, Sebastiaan Mutsaards, Riduan Khaddam Aljameh, Gua Hao Khov, Brecht Machiels, Cristian Olar, Αναστάσιος Ψαρράς, Sander Geursen, Jeroen Vermeeren, Yi Lu, Abhishek Maringanti, Deepak Ameta, Leonidas Katselas, Noah Hütter, Manuel Schmuck, Swetha Sivadas, Karishma Sharma, Manuel Au‐Yong‐Oliveira, Ramon Aerne, Nitish Sharma, Timir Soni, Beatrice Bussolino, Djordje Pesut, Michele Pallaro, Andrei Podlesnii, Alexios Lyrakis, Yannick Ruffiner, Martino Dazzi, Johannes Thiele, Koen Goetschalckx, Nazareno Bruschi, Jonas Doevenspeck, B.A.W. Verhoef, Stefan Linz, Giuseppe Garcea, Jonathan Ferguson, Ioannis Koltsidas, Evangelos Eleftheriou
Abstract
The Metis AI Processing Unit (AIPU) is a quad-core System-on-Chip (SoC) designed for edge inference, executing all components of an AI workload on-chip. The Metis AIPU exhibits performance of 52.4 TOPS per AI core, and a compound throughput of 209.6 TOPS. Key features of the Metis AIPU and its integration into a PCIe card-based system are shown in Fig. 11.3.1. Metis leverages the benefits from a quantized digital in-memory computing (D-IMC) architecture — with 8b weights, 8b activations, and full-precision accumulation — to decrease both the memory cost of weights and activations and the energy consumption of matrix-vector multiplications (MVM), without compromising the neural network accuracy.