Deep Reinforcement Learning on Autonomous Driving Policy With Auxiliary Critic Network
Yuanqing Wu, Siqin Liao, Xiang Liu, Zhihang Li, Renquan Lu
Abstract
Deep reinforcement learning (DRL) is a machine learning method based on rewards, which can be extended to solve some complex and realistic decision-making problems. Autonomous driving needs to deal with a variety of complex and changeable traffic scenarios, so the application of DRL in autonomous driving presents a broad application prospect. In this article, an end-to-end autonomous driving policy learning method based on DRL is proposed. On the basis of proximal policy optimization (PPO), we combine a curiosity-driven method called recurrent neural network (RNN) to generate an intrinsic reward signal to encounter the agent to explore its environment, which improves the efficiency of exploration. We introduce an auxiliary critic network on the original actor-critic framework and choose the lower estimate which is predicted by the dual critic network when the network update to avoid the overestimation bias. We test our method on the lane- keeping task and overtaking task in the open racing car simulator (TORCS) driving simulator and compare with other DRL methods, experimental results show that our proposed method can improve the training efficiency and control performance in driving tasks.