Deep Reinforcement Learning for Cognitive Radar Spectrum Sharing: A Continuous Control Approach
Shane Flandermeyer, Rylee G. Mattingly, Justin G. Metcalf
Abstract
The growing demand for RF spectrum has placed considerable strain on radar systems, which must share limited spectrum resources with an ever-increasing number of devices. It is necessary to design radar systems with coexistence in mind so that the radar avoids harmful mutual interference that compromises the quality of service for other users in the channel. This work presents a deep reinforcement learning (RL) approach to spectrum sharing that enables a pulse-agile radar to operate in heavily congested spectral environments. A cognitive agent dynamically adapts the radar waveform to trade off collision avoidance, bandwidth utilization, and distortion losses due to pulse-agile behavior. Unlike existing RL approaches, this method formulates waveform parameter selection as a continuous control task, significantly increasing the flexibility of the agent and making it possible to scale its behavior to wideband, high-resolution operation. The RL agent uses a recurrent attention-based neural network to select actions, making it suitable for parallelized, real-time implementation. The proposed algorithm makes minimal assumptions about the spectral environment or other users in the spectrum, and the performance of the approach is evaluated on over-the-air data collected from a USRP X310 software-defined radio (SDR) system. Through these experiments, it is shown that the RL approach provides a flexible method for solving multi-objective waveform design problems in dynamic, high-dimensional spectrum environments.