Joint control of traffic signal phase sequence and timing: a deep reinforcement learning method
Zhanbo Sun, Xinben Jia, Yiming Cai, Ang Ji, Xia Lin, Lin Liu, Wenjun Wang, Yuexuan Tu
Abstract
Recent advances in artificial intelligence have opened up new possibilities for optimizing traffic operations. In this study, a novel deep reinforcement learning-based traffic signal control strategy is proposed to jointly optimize phase sequence and signal timing, allowing for more efficient and flexible signal control. A comparison between the proposed approach and two traditional methods, Webster and MaxPressure, is conducted to highlight the advantages of AI-empowered signal control. Different techniques in state representations and action selections are explored to enhance the performance of the DRL-based agent. Results from simulation experiments indicate that the 3DQN framework with prioritized experience replay outperforms other methods by at least a 7.56% queue length reduction. Additionally, the combined state representation of macroscopic feature-based and microscopic cell-based information presents a valuable enhancement for model performance. The ablation experiments demonstrate that considering microscopic information only leads to a 2.44% increase in queue length compared to the proposed method with combined micro-level and macro-level information. However, depending only on the macroscopic feature-based representation, it fails to converge during the training session. Furthermore, the proposed joint control method reduces queue length by 6.37% compared to the phase switching control method, while the single-agent model that optimizes the phase duration performs even worse. Hopefully, this study can offer references for future research in deep reinforcement learning-based traffic signal control schemes and reveal their potential to cope with more dynamic and complex scenarios.