NN-Based Reinforcement Learning Optimal Control for Inequality-Constrained Nonlinear Discrete-Time Systems With Disturbances
Shu Li, Liang Ding, Miao Zheng, Z.Y. Liu, Xinyu Li, Huaiguang Yang, Haibo Gao, Zongquan Deng
Abstract
Based on actor-critic neural networks (NNs), an optimal controller is proposed for solving the constrained control problem of an affine nonlinear discrete-time system with disturbances. The actor NNs provide the control signals and the critic NNs work as the performance indicators of the controller. By converting the original state constraints into new input constraints and state constraints, the penalty functions are introduced into the cost function, and then the constrained optimal control problem is transformed into an unconstrained one. Further, the relationship between the optimal control input and worst-case disturbance is obtained using the Game theory. With Lyapunov stability theory, the control signals are ensured to be uniformly ultimately bounded (UUB). Finally, the effectiveness of the control algorithms is tested through a numeral simulation using a third-order dynamic system.