Efficient Low-Bit Neural Network With Memristor-Based Reconfigurable Circuits
He Xiao, Xiaofang Hu, Tongtong Gao, Yue Zhou, Shukai Duan, Yiran Chen
Abstract
As neural network models are developed and optimized, the use of neural networks in edge devices is increasing, where low-bit neural networks, such as binary neural networks and mixed-precision neural networks, are ideal for edge AI applications. Peripheral circuits and in-memory computing macro are the main components for deploying low-bit precision neural networks on edge AI. However, existing peripheral circuits, including communication units, control modules and analog-to-digital converters (ADCs), are implemented by software or mixed-signal circuits, resulting in significant power and area overheads. To address this issue, memristor-based reconfigurable circuits are proposed for a fully analog implementation of low-bit neural networks without ADCs. In addition, a memristor-based mixed-precision network with a variety of mixed-precision modes is illustrated to verify the effectiveness of deploying low-bit neural networks on edge devices based on the proposed circuits. Furthermore, hybrid simulation results demonstrate that the proposed memristor-based mixed-precision network achieves 84.8 87.5% accuracy on the CIFAR-10 dataset, and the parameter scale of the network model is reduced by 1.6 20x. The circuit analysis demonstrated that the proposed circuits are accurate, robust, and energy-efficient with varying mixed precision, providing a promising and universal solution for applying low-bit neural networks on edge devices.