Mixed-Timescale Request-Driven User Association, Trajectory and Radio Resource Control for Cache-Enabled Multi-UAV Networks
Shuqi Chai, Vincent K. N. Lau
Abstract
Nowadays, unmanned aerial vehicles (UAVs) are being utilized to extend the coverage and capacity of terrestrial wireless networks by exploiting the more favorable line-of-sight (LOS) propagation. The joint optimization of UAV trajectory, user association and radio resource control remains unsolved under a realistic channel model and the more strict QoS requirement in UAV caching networks. Hence, we propose a two-timescale request-driven user association as well as UAVs trajectory and radio resource optimization by exploiting multi-UAV <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$multicasting$</tex-math></inline-formula> <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$opportunities$</tex-math></inline-formula> . The short-term UAV trajectory and radio resource control is adaptive to fast-timescale processes such as channel state information (CSI), queue state information (QSI) and energy state information (ESI), while the long-term user association is adaptive to slow-timescale processes such as the user request profile and user mobility. Based on the timescale decomposition, we derive an online reinforcement learning algorithm to obtain the structural short term optimal policy for the fast-timescale sub-problem, and a parameterized policy gradient learning algorithm to obtain the long term optimal policy for the slow-timescale sub-problem. Finally, simulation results compare the proposed solution to the existing baselines and it is shown that the proposed one can achieve significant gains over all the baselines.