Real-time signal processing enabled by fused networks on a memristor-based system on a chip
Zixu Wang, Zixu Wang, Wenhao Song, Tong Wang, Zihan Wang, Zihan Wang, Yichun Xu, Mingyi Rao, Fuxi Cai, Wenbo Yin, Mike Shuo‐Wei Chen, Ning Ge, Maxwell Collins, Kangjun Bai, Sabyasachi Ganguli, Michael R. Page, Qing Wu, L.P.B. Katehi, Qiangfei Xia, Miao Hu, J. Joshua Yang
Abstract
The von Neumann bottleneck has led to a substantial rise in energy consumption of computing hardware and memory systems, particularly for data-intensive tasks like signal processing. Memristor-based in-memory computing offers an efficient alternative by performing computations within analog memory. Here, we demonstrate real-time signal processing using a fused network that combines the real-time discrete Fourier transform (DFT) and convolutional neural network (CNN) on a memristor-based analog system on a chip (SoC). A 128-by-128 memristor crossbar array performs the DFT on audio signals with a peak signal-to-noise ratio of 33.49 dB, while the following CNN classifies the resulting spectrograms with 94.72% accuracy on the AudioMNIST dataset. In addition, convolution-based edge detection is applied to real-time video frames. The SoC offers substantial energy efficiency improvement over traditional digital systems in signal processing tasks. This work highlights the potential of memristor-based SoCs for efficient real-time signal processing.