Bandwidth Allocation and Trajectory Control in UAV-Assisted IoV Edge Computing Using Multiagent Reinforcement Learning
Juzhen Wang, Xiaoli Zhang, Xingshi He, Yongqiang Sun
Abstract
The rapid development of an unmanned aerial vehicle (UAV) has brought new opportunities for wireless communication and edge computing. In this article, we investigate the scenario where multiple UAVs serve as edge computing devices for the Internet of Vehicles (IoV). Regardless of the allocation of computing resources, we focus on bandwidth allocation and trajectory control to maximize the communication capacity of the system so that the UAV edge computing network can process more data. With this intent, a UAV-assisted IoV edge computing system model is constructed as a nonconvex optimization problem, aiming to maximize the achievable channel capacity of the network. To solve this problem, two “quasi-distributed” multiagent algorithms, i.e., actor-critic mixing network (AC-Mix) and multi-attentive agent deep deterministic policy gradient (MA2DDPG), are proposed based on deep deterministic policy gradient. Specifically, AC-Mix utilizes a mixing network to obtain a global <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$Q$</tex-math></inline-formula> -value for better evaluation of joint action, while MA2DDPG employs a multihead attention mechanism to achieve multiagent collaboration. Using multi-agents deep deterministic policy gradient (MADDPG) as benchmark, several experiments are carried out to verify the performance of the proposed algorithms. Simulation results show that the convergence velocity of AC-Mix and MA2DDPG is improved by 30.0% and 63.3%, respectively, compared with MADDPG.