Safe Reinforcement Learning for Zero-Sum Games of Hypersonic Flight Vehicles
Lei Shi, Xuesong Wang, Yuhu Cheng
Abstract
This article presents a safe reinforcement learning algorithm for the zero-sum game (ZSG) problem of hypersonic flight vehicles within the actor-critic-disturbance framework. Initially, a system transformation on the basis of barrier-function is suggested in which an original safe control issue with full-state constraints is transformed into an equivalent unconstrained optimization problem. Then, an actor-critic-disturbance structure for adaptive optimal learning is proposed to solve the ZSG issue online while assuring safety and stability. Furthermore, based on experience replay technique, novel learning rules for network weights are presented, which not only enable the convergence process of network weights to be more stable, but also accelerate the convergence speed. Thereafter, the stability of the closed-loop system and uniform ultimate boundedness of weight estimation errors are demonstrated by the Lyapunov method. Ultimately, a simulation example is executed to verify the efficiency of the suggested safe reinforcement learning algorithm.