Learning to Schedule Joint Radar-Communication With Deep Multi-Agent Reinforcement Learning
Joash Lee, Dusit Niyato, Yong Liang Guan, Dong In Kim
Abstract
Radar detection and communication are two essential sub-tasks for the operation of next-generation autonomous vehicles (AVs). The forthcoming proliferation of faster 5G networks utilizing mmWave has raised concerns on interference with automotive radar sensors, which has led to a body of research on Joint Radar-Communication (JRC). This paper considers the problem of time-sharing for JRC, with the additional simultaneous objective of minimizing the average age of information (AoI) transmitted by a JRC-equipped AV. We first formulate the problem as a Markov Decision Process (MDP). We then propose a more general multi-agent system, with an appropriate medium access control (MAC) protocol, which is formulated as a partially observed Markov game (POMG). To solve the POMG, we propose a multi-agent extension of the Proximal Policy Optimization (PPO) algorithm, along with algorithmic features to enhance learning from raw observations. Simulations are run with a range of environmental parameters to mimic variations in real-world operation. The results show that the chosen deep reinforcement learning methods allow the agents to obtain strong performance with minimal a priori knowledge about the environment.