Spatial Anti-Jamming Scheme for Internet of Satellites Based on the Deep Reinforcement Learning and Stackelberg Game
Chen Han, Liangyu Huo, Xinhai Tong, Haichao Wang, Xian Liu
Abstract
The anti-jamming communication of the heterogeneous Internet of Satellites (IoS) has drawn increasing attentions due to the smart jamming and high dynamics. This paper investigates a spatial anti-jamming scheme for IoS, with the aim of minimizing anti-jamming routing cost via Stackelberg game and reinforcement learning. Firstly, we formulate the routing anti-jamming problem as a hierarchical anti-jamming Stackelberg game. It has been proved that there is a Stackelberg equilibrium (SE) in the proposed game. Secondly, the spatial anti-jamming scheme for IoS consists of two stages: the available routing selection and the fast anti-jamming decision. To tackle the high dynamics caused by the unknown interrupts and the unexpected congestion, we propose a deep reinforcement learning based routing algorithm (DRLR) to obtain an available routing subset; Furthermore, to make a fast anti-jamming decision, we propose a fast response anti-jamming algorithm (FRA) based on the available routing subset. The user utilizes DRLR and FRA algorithms to empirically analyze the jammer's strategies and adaptively make an anti-jamming decision according to the dynamic and unknown jamming environment. Finally, the simulations have shown that the proposed algorithm has lower routing cost and better anti-jamming performance than existing approaches, and the anti-jamming policies converge to the SE.