Intra-Domain Knowledge Reuse Assisted Reinforcement Learning for Fast Anti-Jamming Communication
Quan Zhou, Yingtao Niu, Xiang Peng, Yonggui Li
Abstract
The novel technology based on reinforcement learning (RL) is considered as a promising direction to achieve cognitive and even intelligent anti-jamming communication. However, traditional RL converges slowly in the face of huge state-action space, while the performance improvement of deep RL depends on high-cost computing resources. Under unknown and dynamic malicious jamming attacks, it is still a major challenge to realize online anti-jamming communication through fast and autonomous learning. In this paper, we propose a fast anti-jamming scheme based on intra-domain knowledge reuse for distributed wireless networks. First, we introduce the Bisimulation Relation to measure the similarity of different state-action pairs in the communication anti-jamming problem, so as to build a bridge of knowledge reuse between state-action pairs. This similarity stems from the fact that the anti-jamming problem can be divided into two categories according to whether it successfully resists jamming attacks, in which different state-action pairs in the same category have similarities. Then, through the real-time update and reuse of state-action values, each node can independently learn the characteristics of dynamic environment and adjust the transmission strategy quickly, so as to avoid external malicious jamming and internal mutual interference. The simulation results show that our proposed scheme significantly accelerates the convergence speed and improves the network normalized throughput compared with the benchmark against unknown and dynamic jamming, and has remarkable agility and jamming memory.