Safe Reinforcement Learning-Based Robust Approximate Optimal Control for Hypersonic Flight Vehicles
Lei Shi, Xuesong Wang, Yuhu Cheng
Abstract
In this paper, a safe reinforcement learning-based robust approximate optimal controller (SRL-RAOC) is proposed with the framework of actor-critic for hypersonic flight vehicles. First, we develop a system transformation based on barrier function, which transforms the issue of full-state safety constraints into an unconstrained optimization problem. Next, the safe reinforcement learning algorithm is employed to design the approximate optimal controller under an online actor-critic framework. Further, a robustifying term is presented to compensate for the neural network approximation errors introduced by the actor-critic framework. Hereafter, the stability analysis of the closed-loop system is accomplished by utilizing the Lyapunov technique, which shows that SRL-RAOC can guarantee that the equilibrium point is asymptotically stable. Moreover, the control input obtained by SRL-RAOC is demonstrated to be close to the optimal control input within a small bound. Finally, a numerical example is provided to demonstrate the effectiveness of SRL-RAOC.