An Efficient and Fast Softmax Hardware Architecture (EFSHA) for Deep Neural Networks
Muhammad Awais Hussain, Tsung‐Han Tsai
Abstract
Deep neural networks are widely used in computer vision applications due to their high performance. However, DNNs involve a large number of computations in the training and inference phase. Among the different layers of a DNN, the softmax layer has one of the most complex computations as it involves exponent and division operations. So, a hardware-efficient implementation is required to reduce the on-chip resources. In this paper, we propose a new hardware-efficient and fast implementation of the softmax activation function. The proposed hardware implementation consumes fewer hardware resources and works at high speed as compared to the state-of-the-art techniques.
Topics & Concepts
Softmax functionComputer scienceComputationComputer architectureHardware architectureArtificial neural networkInferenceComputer hardwareDivision (mathematics)SpeedupComputer engineeringArtificial intelligenceParallel computingSoftwareAlgorithmArithmeticProgramming languageMathematicsAdvanced Neural Network ApplicationsCCD and CMOS Imaging SensorsAdversarial Robustness in Machine Learning