Multiagent Reinforcement Learning in Controlling Offloading Ratio and Trajectory for Multi-UAV Mobile-Edge Computing
Wonseok Lee, Taejoon Kim
Abstract
In this article, a multiunmanned aerial vehicle-mobile-edge computing (UAV-MEC) network is proposed for mobile devices (MDs) located far from a terrestrial base station (BS)-MEC. In the UAV-MEC network, the MDs offload tasks to the UAV-MECs, followed by task offloading from the UAV-MECs to the BS-MEC. Each UAV-MEC aims to jointly optimize its energy consumption, queue stability, and the energy consumption of the MDs by controlling its trajectory and offloading ratio. Specifically, the trajectories of the UAV-MECs are controlled to locate themselves at the optimal positions where the transmission energy of the MDs is minimized. Additionally, the UAV-MECs consider the constraints of task processing time and queue stability in determining the amount of task to be offloaded. Thus, an independent proximal policy optimization (IPPO)-based offloading and trajectory control learning model (IM) is proposed to solve these problems, and a convex optimization model (CM) is also presented to show the optimality of the proposed IM. Specifically, in IM, a near-optimal trajectory control for the randomly located MDs is enabled by exploiting channel gain information only without resorting to accurate location information of the MDs. Furthermore, a dynamic offloading ratio control of each UAV-MEC is achieved by considering its queue dynamics. The simulation results show that the proposed IM jointly optimizes energy consumption and queue stability. Consequently, it outperforms other deep reinforcement learning (DRL)-based algorithms by achieving low energy consumption and long service duration. Moreover, it achieves similar performance to CM while requiring remarkably less time on action decision.