Computing Over the Sky: Joint UAV Trajectory and Task Offloading Scheme Based on Optimization-Embedding Multi-Agent Deep Reinforcement Learning
Xuanheng Li, Xinyang Du, Nan Zhao, Xianbin Wang
Abstract
Unmanned aerial vehicle (UAV)-assisted mobile edge computing (MEC) has emerged to support computation-intensive tasks in 6G systems. Since the battery capacity of a UAV is limited, to serve as many users as possible, a joint design on UAV trajectory and offloading strategy with consideration for service fairness is essential to provide energy-efficient computation offloading to the users in UAV-MEC networks. Unfortunately, such a joint decision-making problem is not straightforward due to various task types required from users and various functionalities of different UAVs enabled by different application programs. Considering the above issues, we take energy efficiency and service fairness as the objective, and propose a <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M</u> ulti- <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">A</u> gent <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">E</u> nergy- <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">E</u> fficient joint <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">T</u> rajectory and <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">C</u> omputation <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</u> ffloading (MA-ETCO) scheme. To adapt to dynamic demands of users, we develop an optimization-embedding multi-agent deep reinforcement learning (OMADRL) algorithm. Each UAV autonomously learns the trajectory control decision based on MADRL to adapt to dynamic demands. Then, it will obtain the optimal computation offloading decision by solving a mixed-integer nonlinear programming problem. The computation offloading result, in turn, will be used as an indicator to guide UAVs’ trajectory design. Compared to relying solely on deep reinforcement learning, such an optimization-embedding way reduces action space dimension and improves convergence efficiency.