An Intelligent Strategy Decision Method for Collaborative Jamming Based on Hierarchical Multi-Agent Reinforcement Learning
Wenxu Zhang, Tong Zhao, Zhongkai Zhao, Yajie Wang, Feiran Liu
Abstract
Aiming at the problem of intelligent cooperative jamming decision-making against frequency agility and frequency diversity in cognitive electronic warfare, an intelligent cooperative jamming strategy decision-making method based on hierarchical multi-agent reinforcement learning is proposed. The multi-agent markov decision process (MDP) is used to construct the multi-jammer cooperative decision-making process. The cooperative jamming decision-making in frequency domain (FD-CJDM) model is established. The design idea of hierarchical reinforcement learning (HRL) is introduced. In order to find the optimal strategy, a double-depth Q-network based on the prioritized experience replay (PER-DDQN) optimization method of sum tree structure is adopted. The performance of FD-CJDM model based on PER-DDQN is simulated. Simulation results show that the proposed PER-DDQN method is obviously superior to deep Q network (DQN) method in action estimation, and its convergence performance is faster than that of double-depth Q network (DDQN). In addition, the intelligent decision-making method of cooperative jamming proposed in this paper can fomulate the frequency domain parameter decision-making strategy according to the order of real-time detected radar threats, which effectively realizes the design of intelligent decision-making in frequency domain.