Neuromorphic Computing in the Era of Large Models
Haoxuan Shan, Chiyue Wei, Nicolas Ramos, Xiaoxuan Yang, Cong Guo, Hai Li, Yiran Chen
Abstract
The rapid advancement of deep learning and the emergence of large-scale neural models, such as bidirectional encoder representations from transformers (BERT), generative pre-trained transformer (GPT), and large language model Meta AI (LLaMa), have brought significant computational and energy challenges. Neuromorphic computing presents a biologically inspired approach to addressing these issues, leveraging event-driven processing and in-memory computation for enhanced energy efficiency. This survey explores the intersection of neuromorphic computing and large-scale deep learning models, focusing on neuromorphic models, learning methods, and hardware. We highlight transferable techniques from deep learning to neuromorphic computing and examine the memory-related scalability limitations of current neuromorphic systems. Furthermore, we identify potential directions to enable neuromorphic systems to meet the growing demands of modern AI workloads.