A COLREGs-Compliant Collision Avoidance Decision Approach Based on Deep Reinforcement Learning
Weiqiang Wang, Liwen Huang, Kezhong Liu, Xiaolie Wu, Jingyao Wang
Abstract
It is crucial to develop a COLREGs-compliant intelligent collision avoidance system for the safety of unmanned ships during navigation. This paper proposes a collision avoidance decision approach based on the deep reinforcement learning method. A modified collision avoidance framework is developed that takes into consideration the characteristics of different encounter scenarios. Hierarchical reward functions are established to assign reward values to constrain the behavior of the agent. The collision avoidance actions of the agent under different encounter situations are evaluated on the basis of the COLREGs to ensure ship safety and compliance during navigation. The deep Q network algorithm is introduced to train the proposed collision avoidance decision framework, while various simulation experiments are performed to validate the developed collision avoidance model. Results indicate that the proposed method can effectively perform tasks that help ships avoid collisions in different encounter scenarios. The proposed approach is a novel attempt for intelligent collision avoidance decisions of unmanned ships.