A Fast Approach to Satellite Range Rescheduling Using Deep Reinforcement Learning
Jun Li Liang, Jianping Liu, Qing Sun, Yuehe Zhu, Yichuan Zhang, Jian-Guo Song, Bo-yong He
Abstract
Real-time emergency event processing for mega-constellation management requires an approach that can quickly obtain a satisfactory satellite range rescheduling scheme. In this article, a deep reinforcement learning (DRL) approach is proposed for rapid rescheduling on the basis of established satellite range scheduling schemes. Three rescheduling principles with different impacts on established scheduling schemes are first analyzed. A mathematical model for both maximizing the completion profits of dynamic missions and minimizing their impacts on scheduled missions is formulated. An efficient DRL-based approach to satellite range rescheduling is then developed. A mask processing mechanism is integrated into the reinforcement learning framework to effectively eliminate the conflicts accurately expressed by the conflict space, and a modified pointer network (Ptr-Net) based on the mask processing mechanism is employed to approximate the action-value function for optimal decision-making. Experimental results show that the proposed approach significantly outperforms the rule-based heuristic and another DRL-based method in terms of computational efficiency and optimization performance, and the total time spent during the training and testing phases of the proposed approach is approximately 13% of the total running time required by the search-based methods.