Deploying LLM Transformer on Edge Computing Devices: A Survey of Strategies, Challenges, and Future Directions
Endah Kristiani, Vinod Kumar Verma, Yang Chao-Tung
Abstract
The intersection of edge computing, Large Language Models (LLMs), and the Transformer architecture is a very active and fascinating area of research. The core tension is that LLMs, which are built on the Transformer architecture, are massive and computationally intensive, while edge devices are resource-constrained in terms of power, memory, and processing capabilities. Therefore, LLMs based on the Transformer architecture are inherently unsuitable for edge computing in their original, full-sized form. They were designed for powerful, resource-rich cloud data centers. However, there is a massive and growing effort to make them suitable for edge devices. Implementing Transformer-based LLMs on edge computing devices is a complex but crucial task that requires a multi-faceted strategy. This paper reviews LLM deployment strategies for Transformer models on edge computing devices, examines the challenges, and estimates future directions. To address these challenges, researchers are exploring methods to compress LLMs and optimize their inference capabilities, making them more efficient for edge environments. Recent advancements in compact LLMs have shown promise in enhancing their deployment on edge devices, enabling improved performance while addressing the limitations of traditional models. This approach not only reduces computational costs but also enhances user privacy and security.