User association and power allocation for UAV-assisted networks: A distributed reinforcement learning approach
Xin Guan, Yang Huang, Chao Dong, Qihui Wu
Abstract
Unmanned aerial vehicles (UAVs) can be employed as aerial base stations (BSs) due to their high mobility and flexible deployment. This paper focuses on a UAV-assisted wireless network, where users can be scheduled to get access to either an aerial BS or a terrestrial BS for uplink transmission. In contrast to state-of-the-art designs focusing on the instantaneous cost of the network, this paper aims at minimizing the long-term average transmit power consumed by the users by dynamically optimizing user association and power allocation in each time slot. Such a joint user association scheduling and power allocation problem can be formulated as a Markov decision process (MDP). Unfortunately, solving such an MDP problem with the conventional relative value iteration (RVI) can suffer from the curses of dimensionality, in the presence of a large number of users. As a countermeasure, we propose a distributed RVI algorithm to reduce the dimension of the MDP problem, such that the original problem can be decoupled into multiple solvable small-scale MDP problems. Simulation results reveal that the proposed algorithm can yield lower long-term average transmit power consumption than both the conventional RVI algorithm and a baseline algorithm with myopic policies.