Mean Field Reinforcement Learning Based Anti-Jamming Communications for Ultra-Dense Internet of Things in 6G
Ximing Wang, Yuhua Xu, Jin Chen, Chunguo Li, Xin Liu, Dianxiong Liu, Yifan Xu
Abstract
Due to the openness of wireless spectrum, the communication security of the Internet of things (IoT) is under threat from various attacks. Radio jamming, as one of the most typical attacks, can easily disrupt the packet transmission and break the availability of spectrum resources. However, traditional anti-jamming methods, such as frequency hopping spread spectrum, are inapplicable to large-scale IoT scenarios for the drawbacks of preset communication patterns and low spectrum efficiency. For the secure spectrum sharing of ultra-dense IoT, in this paper, we model the multi-agent anti-jamming decision-making problem as a quality of service constrained Markov game. To deal with several advanced jamming techniques such as swept jamming and dynamic jamming, we resort to a model-free multi-agent reinforcement learning (MARL) algorithm, and develop a mean field DeepMellow based anti-jamming method to achieve the Nash equilibrium solution of the game. The simulation results show that the algorithm enables agents to collaboratively share the spectrum and simultaneously avoid the jamming attack, which demonstrates the effectiveness of the proposed algorithm.