One to Any: Distributed Conflict Resolution with Deep Multi-Agent Reinforcement Learning and Long Short-Term Memory
Marc Brittain, Peng Wei
Abstract
View Video Presentation: https://doi.org/10.2514/6.2021-1952.vid A novel deep multi-agent reinforcement learning framework is proposed to identify and resolve conflicts between a variable number of aircraft in a high-density, stochastic, and dynamic en route airspace sector. With a growing demand in air transportation, ensuring these dynamic systems are both safe and efficient is essential. Air traffic is becoming denser and more complex, not only in traditional airspace, but also in low altitude airspace. Therefore, we need an autonomous air traffic control system to ensure safe separation requirements in these environments. We propose a Deep Distributed Multi-Agent Variable framework (D2MAV) that utilizes an actor-critic algorithm, Proximal Policy Optimization (PPO) that incorporates a Long Short-Term Memory (LSTM) network to encode a variable number of aircraft states into a fixed length vector. This allows the agents to have access to all aircraft information in the sector in a scalable, efficient way to achieve high traffic throughput under uncertainty. We train the agents using a centralized learning, decentralized execution scheme where one neural network is learned and shared by all agents in the environment. We evaluate our framework via simulation in the BlueSky air traffic control environment.