Sensor Tasking for Space Situation Awareness: Combining Reinforcement Learning and Causality
Zhengyana Fan, Rocky K. C. Chang, Ali K. Raz, Ashton Harvey, Genshe Chen
Abstract
Tracking resident space objects (RSOs), which include functional and non-functional satellites and space debris, requires flexible and adaptive tasking of sensors that operate in a distributed network. Allocating sensors to RSOs for maintaining track needs a multitude of criteria such as timely access to information, rapid decision making over complex outcomes, and integration between sensors to maintain coverage over extended period of time. Recent studies have shown promising potential of Deep Reinforcement Learning (DRL) for solving the sensor allocation problem (SAP). These DRL solutions, nevertheless, are not transparent and the inner combinatorial complexity of SAP remains uninterpretable. In this paper, we incorporate causality into the DRL approaches in an effort to make DRL based SAP solutions interpretable and explainable. We propose a structural causal model (SCM) of DRL and show that the state space can be reduced and the learning efficiency of DRL agent can be improved. We carry out a series of simulation experiments to illustrate the effectiveness of the simplified model. Performance results show that the simplified DRL agent using causal model can achieve the same level of performance as a non-simplified agent with an added capability of potentially explaining the resulting learning policy.