Integrated algorithm and hardware design for hybrid neuromorphic systems
James Seekings, Mahsa Ardakani, Peyton Chandarana, Arshia Eslami, Mohammadreza Mohammadi, Ramtin Zand
Abstract
This paper investigates the combined potential of neuromorphic and edge computing to develop a flexible machine learning (ML) system designed for processing data from dynamic vision sensors. We build and train hybrid models that integrate spiking neural networks (SNNs) and artificial neural networks (ANNs) using the PyTorch and Lava frameworks. We explore the effects of quantization on ANN models to assess its impact on both accuracy and energy efficiency. Additionally, we address the challenges of deploying hybrid models on hardware by implementing individual components on specific edge platforms. We also propose an accumulator circuit to bridge the spiking and non-spiking domains. Comprehensive performance analyses are conducted on a heterogeneous system of neuromorphic and edge AI hardware, assessing accuracy, latency, and energy consumption. Our results show that hybrid spiking networks improve accuracy and energy efficiency. Moreover, we find that quantization improves hybrid networks, further reducing energy consumption while boosting accuracy.