A hierarchical deep reinforcement learning method for solving urban route planning problems under large-scale customers and real-time traffic conditions
Yuanyuan Li, Yuanyuan Li, Qingfeng Guan, Jun Feng Gu, X. S. Jiang, Li Yang, Li Yang
Abstract
As urbanization and economic growth advance, large-scale customers and real-time traffic conditions have become crucial factors in urban route planning. Deep reinforcement learning is considered the most effective method for solving urban route planning problems involving large-scale customers and real-time traffic conditions. Due to memory usage limitations, existing deep reinforcement learning methods cannot identify candidate customers or determine optimal travel routes in large-scale and real-time environments. To tackle these problems, this study introduces a hierarchical deep reinforcement learning method utilizing an improved transformer model (HDRLITF) based on the divide-and-conquer concept. Graph attention networks and gate mechanisms are integrated into the transformer model to capture dynamic features and improve the model’s performance. A two-stage training method, based on the actor-critic algorithm, is proposed to determine the optimal policy function. To evaluate the HDRLITF method, experiments were conducted using datasets from the cities of Shenzhen and Chengdu in China. The experimental results suggest that the HDRLITF method can effectively interact with real-time traffic environments and obtain high-quality solutions compared to other deep reinforcement learning methods. The robustness and reliability of HDRLITF were further validated across multiple traffic scenarios and indicators.