Human-like route planning for automatic collision avoidance using generative adversarial imitation learning
Takefumi Higaki, Hirotada Hashimoto
Abstract
Automatic collision avoidance systems are expected to eliminate maritime accidents caused by human error. Recent studies have shown that ships can prevent collisions in complex situations by using deep reinforcement learning (DRL). However, rewards must be designed to implement DRL, which can be challenging for ambiguous tasks, such as following the Convention on the International Regulations for Preventing Collisions at Sea (COLREGs). While some studies have used inverse reinforcement learning (IRL) to derive an appropriate reward for limited problems with small, discrete state spaces, this study introduced generative adversarial imitation learning (GAIL) and proposed a route planning algorithm that addresses the above limitations. We applied DRL to generate sample collision avoidance trajectories and demonstrate that the imitative route planner based on GAIL can reproduce these trajectories. Also, expert trajectories were collected in simulation experiments involving a well-experienced captain, and we attempted to generate collision avoidance routes that mimic human expert performance. By applying a convolutional neural network and other techniques to improve the planner's imitation capability, we established a sophisticated route planner that is consistent with captain's risk assessments and tolerant of noise.