Enhancing Robustness of Deep Reinforcement Learning Based Adaptive Traffic Signal Controllers in Mixed Traffic Environments Through Data Fusion and Multi-Discrete Actions
Tianjia Yang, Wei Fan
Abstract
With the rapid development of artificial intelligence (AI) and connected vehicle (CV) technology, researchers are actively exploring the utilization of deep reinforcement learning (DRL) algorithms combined with real-time traffic information from CVs to optimize traffic signal control. These controllers have showcased better performance than traditional controllers. However, a major drawback is their heavy reliance on pure CV environments, which has not been adequately addressed. This study proposes a novel traffic signal controller based on proximal policy optimization (PPO), integrating a multi-discrete action space and a combined state space, to enhance robustness in mixed traffic environments where CVs and non-connected vehicles coexist. Evaluations through simulation experiments on a real-world-based intersection testbed demonstrate superior performance in terms of both effectiveness and robustness compared to some popular controllers, including the deep Q-network (DQN) based controller, pretimed controller, and actuated controller. The results indicate that the proposed controller significantly reduces the average delay. Furthermore, its performance remains reliable even in environments with a CV market penetration rate as low as 20%. The findings highlight that the utilization of PPO with multi-discrete actions and combined state space effectively addresses the challenges posed by mixed traffic environments, making it a promising solution for real-world traffic signal control.