A Federated Reinforcement Learning Approach for Optimizing Wireless Communication in UAV-Enabled IoT Network With Dense Deployments
Fan Yang, Zijie Zhao, Huang Jie, Peifeng Liu, Amr Tolba, Keping Yu, Mohsen Guizani
Abstract
In unmanned aerial vehicle (UAV)-enabled Internet of Things (IoT) networks, the communication ranges between densely deployed IoT devices overlap, resulting in wireless resource conflicts between them. Hence, achieving conflict-free resource allocation is a challenging issue that must be urgently addressed for UAV-enabled IoT networks. To tackle this issue, a hypergraph is used to quantify conflicts, and a federated reinforcement learning (RL)-based resource allocation framework is proposed. Specifically, a conflict graph model is developed for UAV-enabled IoT networks with dense deployments. The model is then converted into a conflict hypergraph model using hypergraph and faction theory. Consequently, the conflict avoidance problem of resource allocation can be reformulated as a hypergraph node coloring problem. The problem is formulated as a Markov decision process, which is solved using a deep RL-based approach. Additionally, to distribute the computational workload across the network and alleviate the burden on the central server, we propose the FedAvg dueling double deep Q-network (FedAvg-D3QN). The proposed FedAvg-D3QN is verified through simulation to have advantages in resource reuse rate and throughput compared to baseline approaches.