PICACHU: Plug-In CGRA Handling Upcoming Nonlinear Operations in LLMs
Jiajun Qin, Tianhua Xia, Cheng Tan, Jeff Zhang, Sai Qian Zhang
Abstract
Large language models (LLMs) have revolutionized natural language processing (NLP) domain by achieving state-of-the-art performance across a range of benchmarks. However, nonlinear operations in LLMs significantly contribute to inference latency and present unique challenges that have not been encountered previously. Addressing these challenges requires accelerators that combine efficiency, flexibility, and support for user-defined precision. Our analysis reveals that Coarse-Grained Reconfigurable Arrays (CGRAs) provide an effective solution, offering a balance of performance and flexibility tailored to domain-specific workloads.
Topics & Concepts
Computer sciencePlug and playPlug-inSpark plugNonlinear systemEngineeringOperating systemMechanical engineeringPhysicsQuantum mechanicsDistributed and Parallel Computing SystemsParallel Computing and Optimization TechniquesAlgorithms and Data Compression