Optimization for Dynamic Laser Inter-Satellite Link Scheduling With Routing: A Multi-Agent Deep Reinforcement Learning Approach
Guanhua Wang, Fang Yang, Jian Song, Zhu Han
Abstract
Laser inter-satellite links (LISLs) have greatly extended communication distance between satellites, allowing for establishment of dynamic links to reduce communication delay. However, a closed-loop control is required for LISL, which causes high energy consumption. Proper scheduling of dynamic LISLs can effectively reduce energy consumption and communication delay. In this study, a satellite link mode with three fixed LISLs and one dynamic LISL is designed, and its feasibility is analyzed. The optimization problem is formulated and transformed into a Markov decision process (MDP) by modeling it as a sequential decision. By decomposing states, actions, and reward functions, the MDP is divided into the proposed multi-agent deep reinforcement learning (MADRL). Moreover, compressed sensing is utilized to cut down state information to reduce communication, storage, and computation overhead. Furthermore, network parameters and experience sharing, and prioritized experience replay have been adopted to improve stability and convergence speed of network training with a large number of agents. Experimental results show that under different routing strategies, the proposed MADRL can reduce energy consumption by over 15% and delay by approximately two hops compared to fixed LISLs scenario within several iterations.