B-APFDQN: A UAV Path Planning Algorithm Based on Deep Q-Network and Artificial Potential Field
Fuchen Kong, Qi Wang, Shang Gao, Hualong Yu
Abstract
Deep Q-network (DQN) is one of the standard methods to solve the Unmanned Aerial Vehicle (UAV) path planning problem. However, the way agent deepens its cognition of the environment through frequent random trial-and-error leads to slow convergence. This paper proposes an optimized DQN with Artificial Potential Field (APF) as prior knowledge called B-APFDQN for path planning. Replacing the traditional neural network which has only one Q-value output with a multi-output neural network to promote the training process in combination with APF. Furthermore, a SA- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\varepsilon $ </tex-math></inline-formula> -greedy algorithm that can automatically adjust the stochastic exploration frequency with steps and successes is proposed in order to prevent the agent from falling into local optimum. We remove the nodes that do not affect the path connectivity and apply the B-spline algorithm to make the path shorter and smoother. Simulation experiments show that the proposed B-APFDQN algorithm performs better than the classical DQN, has a strong ability to avoid falling into local optimum, and the obtained paths are smooth and shorter, which proves the effectiveness of B-APFDQN in the UAV path planning problem.