DFX: A Low-latency Multi-FPGA Appliance for Accelerating Transformer-based Text Generation
Seongmin Hong, Seungjae Moon, Junsoo Kim, Sungjae Lee, Minsub Kim, Dongsoo Lee, Joo-Young Kim
Abstract
•DFX: a low-latency multi-FPGA appliance for accelerating transformer-based text generation–DFX is a multi-FPGA appliance that accelerates transformer-based text generation–DFX adopts model parallelism to efficiently process the large-scale language model–Xilinx Alveo U280 data center accelerator card provides high performance with low-cost–FPGA-to-FPGA communication is enabled by QSFP cable at 100 Gb/s
Topics & Concepts
Field-programmable gate arrayComputer scienceTransformerLatency (audio)Embedded systemComputer hardwareElectrical engineeringEngineeringVoltageTelecommunicationsEmbedded Systems Design TechniquesParallel Computing and Optimization TechniquesAdvanced Data Storage Technologies