Optimal Hovering Control Strategy for UAVs Using PPO
Yutong Luo, Jianing Xu, Yongzhou Chen, Yinuo Yang, Tianyao Zheng, Hao Liu
Abstract
Hovering stability is a fundamental requirement for unmanned aerial vehicles (UAVs) in missions such as surveillance, inspection, and precision delivery. Conventional controllers, such as PID and model-based methods, often face performance degradation under environmental disturbances, sensor noise, and model uncertainties. To address these challenges, this paper presents an optimal hovering control strategy for UAVs using Proximal Policy Optimization (PPO). A comprehensive state-action representation of UAV dynamics is constructed, and a reward function is designed to jointly encourage precise position holding, energy-efficient control, and disturbance rejection. The proposed policy is trained iteratively in a high-fidelity simulation environment until convergence. To validate its effectiveness, extensive experiments compare the PPO-based controller with two state-of-the-art deep reinforcement learning baselines—Deep Deterministic Policy Gradient (DDPG) and Soft Actor-Critic (SAC). Results show that the proposed method achieves higher hovering accuracy, improved robustness under wind gusts and sensor noise, and reduced control effort compared with both conventional and DRL baselines. These findings demonstrate the potential of PPO for high-performance autonomous UAV hovering control and provide a solid foundation for real-world deployment and extension to more complex flight tasks.