A Novel Configurable High-precision and Low-cost Circuit Design of Sigmoid and Tanh Activation Function
Feng Liu, Bowen Zhang, Gang Chen, Guoliang Gong, Huaxiang Lu, Wenchang Li
20212021 IEEE International Conference on Integrated Circuits, Technologies and Applications (ICTA)21 citationsDOI
Abstract
This paper proposes a configurable Sigmoid and Tanh activation function circuit design by second-order approximation and deviation compensation. The proposed circuit is synthesized in TSMC 180nm technology within a 7218 um <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> area at 500MHz, it is also deployed on Xilinx ZC706 FPGA by 158 LUTs and 46 FFs at 333MHz. It achieves almost 4.68× better precision and 20.2% less area than previous works. The experimental results show that our method has almost no accuracy loss on LSTM Chinese speech recognition tasks and the YOLOv3 image recognition post-processing task.
Topics & Concepts
Sigmoid functionHyperbolic functionField-programmable gate arrayComputer scienceTask (project management)Function (biology)Compensation (psychology)Activation functionImage (mathematics)Artificial intelligenceSoftwareComputer hardwareSpeech recognitionAlgorithmMathematicsArtificial neural networkEngineeringProgramming languageBiologySystems engineeringEvolutionary biologyMathematical analysisPsychoanalysisPsychologyNeural Networks and ApplicationsBlind Source Separation TechniquesAnalog and Mixed-Signal Circuit Design