Edge and Terminal Cooperation Enabled LLM Deployment Optimization in Wireless Network

Abstract

Recent years have witnessed the remarkable emer-gence of large language models (LLMs), demonstrating exceptional capabilities in handling complex tasks such as natural language processing, computer vision, and speech processing. However, the prevalent deployment of LLMs on cloud servers poses challenges in ensuring user data privacy and minimizing computation and transmission delay. Moreover, LLM needs to read all the parameters to generate a token for each inference and then splices the original input with the generated token as a new input to generate the next token. To address these concerns, we propose a novel LLM deployment framework leveraging edge and terminal cooperation. Our proposed framework involves deploying LLMs on both edge server and terminal device, where the LLM model on the terminal generates tokens serially, while the model on the edge generates tokens in parallel. This edge-terminal cooperative and serial and parallel combination framework can effectively reduce the computation delay of token generation. To further optimize the performance of the proposed framework, we formulate an optimization problem aimed at minimizing average delay and energy consumption. Considering its integer programming property, we employ a branch and bound algorithm to efficiently solve this problem. Extensive simulation results provide compelling evidence of the effectiveness of our LLM deployment optimization algorithm. Under diverse channel conditions and terminal device computing capabilities, our proposed scheme consistently outperforms other baseline algorithms in terms of delay and energy consumption. These findings underscore the potential of our approach in facilitating the practical deployment of LLMs on edge and terminal.

Topics & Concepts