Load-Aware Distributed Resource Allocation for MF-TDMA Ad Hoc Networks: A Multi-Agent DRL Approach
Shuying Zhang, Zuyao Ni, Linling Kuang, Chunxiao Jiang, Xu Zhao
Abstract
Without control center assistance, it is difficult that users take traffic loads of whole Ad Hoc networks into account to utilize wireless resources collaboratively. Considering influence of load differences between users on resource allocation can be learned in advance, we construct a framework of load-aware distributed resource allocation for multiple frequencies time division multiple access (MF-TDMA) wireless Ad Hoc networks based on multi-agent deep reinforcement learning (DRL) with centralized training and distributed execution. In this framework, each user is regarded as an agent and the wireless transmission resources are partitioned into time-frequency resource blocks (RBs). Each user reserves RBs by sending network manager operational packet (NMOP) in turn at rotating broadcast stage (RBS) and how many RBs each user reserves depends on the corresponding agent's decision. The main innovative point of our work is achieving a result of wireless resource allocation that balance load degree of all users on the whole network in a distributed manner. With defining a novel network utility parameterized by traffic loads (termed timely capacity), we first model the resource allocation as an optimization problem for timely capacity sum maximization. Using the optimization objective as common reward function to train neural networks of all agents in an offline simulation environment, we obtain the distributed algorithm based on multi-agent DRL. Experimental results demonstrate that with the neural networks trained by the designed reward and enough training, users can successfully learn to collaboratively improve network utility in distributed implementation.