A 28nm 77.35TOPS/W Similar Vectors Traceable Transformer Processor with Principal-Component-Prior Speculating and Dynamic Bit-wise Stationary Computing
Yang Wang, Yubin Qin, Dazheng Deng, Xiaolong Yang, Zhiren Zhao, Ruiqi Guo, Zhiheng Yue, Leibo Liu, Shaojun Wei, Yang Hu, Shouyi Yin
Abstract
This paper proposes an energy-efficient Transformer processor exploiting dynamic similarity in global attention computing. It has three features: 1) A principal-component-prior speculation unit (PCSU) removes 28.4% of redundant computations. 2) A similar-vector tracked computing engine (STCE) saves 42.2% of multiplications. 3) A bit-wise stationary processing element (BSPE) reduces multiplication energy by $1.47\times$. The proposed processor achieves a peak energy efficiency of 77.35TOPS/W. It reduces energy by $2.81\times$ and offers $3.71\times$ speedup compared with the state-of-the-art Transformer processor.