An Online Distributed Satellite Cooperative Observation Scheduling Algorithm Based on Multiagent Deep Reinforcement Learning
Dalin Li, Wang Haijiao, Yang Zhen, Yanfeng Gu, Shen Shi
Abstract
The provision of real-time information services is one of the crucial functions of satellites. In comparison with the centralized scheduling, the distributed scheduling can provide better robustness and extendibility. However, the existing distributed satellite scheduling algorithms require a large amount of communication between satellites to coordinate tasks, which makes it difficult to support scheduling in real-time. This letter proposes a multiagent deep reinforcement learning (MADRL)-based method to solve the problem of scheduling real-time multisatellite cooperative observation. The method enables satellites to share their decision policy, but it is not necessary to share data on the decisions they make or data on their current internal state. The satellites can use the decision policy to infer the decisions of other satellites to decide whether to accept a task when they receive a new request for observations. In this way, our method can significantly reduce the communication overhead and improve the response time. The pillar of the architecture is a multiagent deep deterministic policy gradient network. Our simulation results show that the proposed method is stable and effective. In comparison with the Contract Net Protocol method, our algorithm can reduce the communication overhead and achieve better use of satellite resources.