Real-Time UAV Path Planning Based on LSTM Network
Jiandong Zhang, Yukun Guo, Lihui Zheng, Qiming Yang, Guoqing Shi, Yong Wu
Abstract
To address the shortcomings of single-step decision making in the existing deep reinforcement learning based unmanned aerial vehicle (UAV) real-time path planning problem, a real-time UAV path planning algorithm based on long short-term memory (RPP-LSTM) network is proposed, which combines the memory characteristics of recurrent neural network (RNN) and the deep reinforcement learning algorithm. LSTM networks are used in this algorithm as Q-value networks for the deep Q network (DQN) algorithm, which makes the decision of the Q-value network has some memory. Thanks to LSTM network, the Q-value network can use the previous environmental information and action information which effectively avoids the problem of single-step decision considering only the current environment. Besides, the algorithm proposes a hierarchical reward and punishment function for the specific problem of UAV real-time path planning, so that the UAV can more reasonably perform path planning. Simulation verification shows that compared with the traditional feed-forward neural network (FNN) based UAV autonomous path planning algorithm, the RPP-LSTM proposed in this paper can adapt to more complex environments and has significantly improved robustness and accuracy when performing UAV real-time path planning.