A 22nm 8Mb STT-MRAM Near-Memory-Computing Macro with 8b-Precision and 46.4-160.1TOPS/W for Edge-AI Devices

Abstract

Nonvolatile-memory-based computing in memory (nvCIM) [1–6] is ideal for low-power edge-Al devices requiring neural network (NN) parameter storage in the power-off mode, a rapid response to device wake-up, and high energy efficiency for MAC operations <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$(\text{EF}_{\text{MAC}})$</tex> . Current analog nvCIMs impose a tradeoff between the signal margin (SM) and the number of accumulations <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$(\mathrm{N}_{\mathrm{A}\text{CU}})$</tex> per cycle versus <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\text{EF}_{\text{MAC}}$</tex> and computing latency <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$(\mathrm{T}_{\text{CD}-\text{MAC}})$</tex> . Near-memory computing (NMC), with high precision for inputs (IN), weights (W), and outputs (OUT), and a high <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathrm{N}_{\text{ACU}}$</tex> is a trend to improve <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\text{EF}_{\text{MAC}}, \mathrm{T}_{\text{CD}-\text{MAC}}$</tex> , and accuracy. A prior STT-MRAM NMC [1] uses vertical-weight mapping (VWM) to improve the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\text{EF}_{\text{MAC}}$</tex> ; however, further improvement is challenging: due to (1) the large energy consumption in reading repetitious weight data across multiple inputs for a single NN-layer; (2) a high bitstream toggling-rate (BTR) for digital MAC circuits <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$(\text{DC}_{\text{MAC}})$</tex> reduces <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\text{EF}_{\text{MAC}}$</tex> , and; (3) a limited SM and memory readout latency <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$(\mathrm{T}_{\text{CD}-\mathrm{M}})$</tex> for memories with a small R-ratio (e.g. STT-MRAM, see Fig. 33.2.1). In developing an STT-MRAM nvCIM macro, this work moves beyond circuit-level novelty by using system-software-circuit co-design. This work achieves a high <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\text{EF}_{\text{MAC}}$</tex> , a short <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathrm{T}_{\text{CD-M}}$</tex> , a high read bandwidth (R-BW), a high IN-W-OUT precision, and a high <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathrm{N}_{\text{ACU}}$</tex> by using the novel schemes: (1) a hardware based weight-feature aware read (WFAR) to reduce weight accesses and improve <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\text{EF}_{\text{MAC}}$</tex> with a minimal area overhead; (2) toggling-aware weight-tuning (TAWT) to obtain fine-tuned weights <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$(\mathrm{W}_{\text{FT}})$</tex> with a low BTR, which is based on VWM to enhance the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\text{EF}_{\text{MAC}}$</tex> of the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\text{DC}_{\text{MAC}}$</tex> ; (3) a differential charge-accumulating margin-enhanced voltage-sensing amplifier (DCME-VSA) to enhance the SM, while reducing the T <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">CD</inf> - <inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M</inf> . The proposed 22-nm S-Mb STT-MRAM NMC nvCIM macro achieves the highest R-BW <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$(436\text{GB}/\mathrm{s})$</tex> and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\text{EF}_{\text{MAC}}(46.4-160.1\text{TO}\text{PS}/\mathrm{W})$</tex> for <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathrm{N}_{\mathrm{A}\text{CU}}=576$</tex> for SblN - SbW - 26bOUT.

Topics & Concepts