Multiagent Reinforcement Learning for Antijamming Game of Frequency-Agile Radar
Jie Geng, Bo Jiu, Kang Li, Yu Zhao, Chao Wang, Hongwei Liu
Abstract
With the development of jamming systems, the jammer is becoming much smarter and can change its strategy with radar, which forms a competitive game between the radar and jammer and poses a significant threat to the radar. In this paper, the anti-jamming game of frequency-agile (FA) radar is investigated, in which the jammer is capable of transmitting jamming signals with multiple frequency channels and different power allocation manners. To characterize the sequential interaction and partial observation of the anti-jamming game, an extensive-form game is introduced to model the confrontation between the radar and jammer. Subsequently, a novel multi-agent reinforcement learning (MARL) algorithm, termed NFSP-DDPG, is devised by combining neural fictitious self-play (NFSP) with deep deterministic policy gradient (DDPG) and modifying the supervised learning process of NFSP to solve Nash equilibrium (NE) strategies, which can handle the anti-jamming games featuring both discrete and continuous action spaces. Finally, simulation results show that the learning strategies acquired through the proposed method approximate NE and outperform the rule-based strategies. The signal-to-interference-pulse-noise ratio (SINR) improvements of radar taking NE strategy are 9.13dB, 9.12dB, and 9.15dB in the worst-case in the anti-jamming game with five frequencies compared to three rule-based strategies.