A Deep Q-Network Reinforcement Learning-Based Model for Autonomous Driving
Marwa Ahmed, Chee Peng Lim, Saeid Nahavandi
Abstract
Learning to drive in highly crowded urban areas with multi-agent interaction is a challenging task. Most of the current methods rely on hand-crafting the policy required for the decision-making process. In contrast, reinforcement learning (RL) offers proper methodologies to automatically formulate the best policy without manual intervention. However, the baseline RL algorithms face a challenge in handling simulated driving situations (e.g., keep driving in the current lane center, and keep a safe distance from a leading vehicle) in high fidelity and complex urban areas. In this study, we propose an end-to-end autonomous driving system using a Deep Q-Network (DQN) and long-short-term memory (LSTM) with a new observation input to enable the RL agent to learn driving in complex environments i.e. CARLA simulator. The observation input comprises a tuple of RGB (Red, Green, Blue) image data from the forward-facing camera, vehicle speed, and vehicle angle from the road center. The output comprises the steering, brake, and acceleration commands. Our proposed model, in conjunction with formulating the proper reward function, results in a rapid convergence with a better performance and safer driving behaviors from an autonomous driving agent. The results indicate that the LSTM-DQN model enables the autonomous agent in controlling the vehicle in a lane while keeping a safe distance from the leading vehicle in varied unseen road conditions.