Target Tracking Control of UAV Through Deep Reinforcement Learning
Bodi Ma, Zhenbao Liu, Wen Zhao, Jinbiao Yuan, Hao Long, Xiao Wang, Zhirong Yuan
Abstract
This study presents an innovative reinforcement-learning-based control algorithm for a vertical take-off and landing (VTOL) aircraft under wind disturbances. In our approach, the tracking control problem of the VTOL aircraft is formulated as a Markov decision process, and the appropriate system state, reward function, and soft update method are presented. To improve the control accuracy under wind disturbances, three kinds of wind fields were added in the learning environment to expand the exploration space and simulate the effect of wind disturbances on the flight control. Moreover, to ensure the tracking accuracy and the practical implementation, a quantum-inspired experience replay strategy was developed based on quantum computation theory. In this strategy, the preparation operation scheme was designed to encourage the exploration and speed up the convergence. The depreciation operation method was developed to enrich the sample diversity, which increased the robustness of the controller and allowed the control strategy learned in the numerical simulations to be directly transferred into real-world environments. Numerical simulations, hardware-in-the-loop experiments, and real-world flight experiments were conducted to evaluate the performance and merits of the proposed method. The results demonstrated high accuracy and effectiveness and good robustness of the proposed control algorithm in terms of standoff target tracking control and flight stability.