Deep Reinforcement Learning for Channel Traffic Scheduling in Dry Bulk Export Terminals
Wenyuan Wang, Ao Ding, Zhen Cao, Yun Peng, Huakun Liu, Xinglu Xu
Abstract
Heavy navigation demands of incoming/outgoing ships and operational features in dry bulk export terminals (DBETs) call for effective and intelligent optimization methods to improve navigation channel traffic and reduce delays. Considering ship deballasting delays in DBETs and their influences on channel traffic flow, this paper proposes a channel traffic scheduling (CTS) optimization method based on deep reinforcement learning (DRL). The CTS problem is formulated into a Markov decision process with tailored state and action definitions. Practical constraints, such as tidal windows and dynamic switching traffic mode, are incorporated into action selection processes. In coping with large state-action spaces caused by practical applications, a hierarchical DRL framework is proposed to perform layered decision-making. Relying on the reward signal design, DRL agents can learn optimization policies to produce integrated scheduling plans of channel traffic and ship deballasting operations while minimizing ship mooring, unberthing, and deballasting delays. A proximal policy optimization method is developed for coupling training DRL agents. Numerical experiments demonstrate that the proposed method can converge faster to better scheduling strategies and efficiently generate high-quality CTS solutions for industrial-scale applications.