Transformer-OPU: An FPGA-based Overlay Processor for Transformer Networks
Yueyin Bai, Hao Zhou, Keqing Zhao, Jianli Chen, Jun Yu, Kun Wang
Abstract
Existing implementations of transformer networks by field-programmable gate array (FPGA) focus only on attention computation, or suffer from fixed model structure without flexibility. In this article, we propose an FPGA-based overlay processor, named Transformer-OPU for general accelerations of transformer networks. Experimental result shows that our Transformer-OPU achieves 5.19-15.06× and 1.14-2.89× speedup compared with CPU and GPU, respectively. We also observe 1.10-2.47× better latency compared with previously customized FPGA accelerators, and is 1.45× faster than NPE.
Topics & Concepts
Field-programmable gate arrayComputer scienceSpeedupOverlayComputationTransformerParallel computingEmbedded systemElectrical engineeringEngineeringVoltageOperating systemAlgorithmVLSI and FPGA Design TechniquesOptimal Power Flow DistributionParallel Computing and Optimization Techniques