Research on vertical strategy for left turn at signal-free T-shaped intersections based on multi-layer reinforcement learning methods

Abstract

The task of executing left turns at signal-free T-shaped intersections without protective signals poses a critical challenge in the realm of autonomous driving. Conventional rule-based approaches tend to be excessively cautious, rendering them inadequate for effectively managing driving tasks within unpredictable T-shaped intersection environments. In the case of complex traffic scenarios, a single model is less effective in convergence and has a lower pass rate and poorer safety. Thus, this study introduces a multi-layer reinforcement learning model, employing D3QN(Dueling Double DQN) and TD3(Twin Delayed Deep Deterministic policy gradient algorithm) for advanced behavioral decision-making and vertical acceleration planning, respectively. In our experimental investigation, we designed four simulation scenarios based on the driving behavior of the Carla simulator to replicate real-world driving conditions. Verification and test simulation outcomes substantiate that, in comparison to other single-trained reinforcement learning models, the multi-layer reinforcement learning model proposed in this study attains the highest success rate. Specifically, the pass rate in the verification scenario, consistent with the training conditions, achieves an impressive 99.5%. Furthermore, the pass rate in the comprehensive test scenario reaches 89.6%. These experiments unequivocally demonstrate the considerable enhancement in T-shaped intersections pass rates achieved by the proposed method while ensuring both traffic efficiency and safety. • Pre-generating vehicle trajectory points and training vehicle longitudinal acceleration parameters using a reinforcement learning approach. • Adopting a multi-layer reinforcement learning approach, we utilize an upper-level discrete reinforcement learning method to train a pre-trained continuous reinforcement learning model, and feed the results into the environment. • The proposed method in this paper, when compared to the original single reinforcement learning approach, demonstrates improved safety while ensuring efficiency across four different oncoming traffic scenarios.

Topics & Concepts