Computing with magnetic tunnel junction based sigmoidal activation functions
Youwei Bao, Shuhan Yang, Zhaoyang Yao, Hyunsoo Yang
Abstract
Nonlinear activation functions play a crucial role in artificial neural networks. However, digital implementations of sigmoidal functions, the commonly used activation functions, are facing challenges related to energy consumption and area requirements. To address these issues, we develop a proof-of-concept computing system that utilizes magnetic tunnel junctions as the key element for implementing sigmoidal activation functions. Using this system, we train a neural network for speech separation. When compared to state-of-the-art digital implementations, our scalable circuit has the potential to consume up to 383 times less energy and occupy 7354 times smaller area. These results pave the way for more efficient computing systems in the future.