Adaptive Optimal Control via <i>Q</i>-Learning for Itô Fuzzy Stochastic Nonlinear Continuous-Time Systems With Stackelberg Game

Abstract

In order to solve the two-player Stackelberg game for the continuous-time nonlinear stochastic system, using the Takagi–Sugeno (T-S) fuzzy stochastic model, this paper defines the novel <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$Q$</tex-math></inline-formula> -functions and suggests an adaptive dynamic programming (ADP)-based approach that is completely model-free. First, based on the T-S fuzzy model, the overall fuzzy control policies with corresponding cost functions are designed where coupled penalty functions are considered. Subsequently, we create a novel two-level algorithm based on integral reinforcement learning and provide the proof of convergence to overcome challenge of computing the optimal cost functions analytically. On this basis, in order to achieve entirely model-free learning, which is the first attempt in solving fuzzy stochastic nonlinear continuous-time systems with the Stackelberg game problem, the innovative action-dependent <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$Q$</tex-math></inline-formula> -functions are developed. Fuzzy linearization technique and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$Q$</tex-math></inline-formula> -learning algorithm are ingeniously combined in this article to solve their respective difficulties. In addition, the Lyapunov approach under the ADP-based control scheme ensures the stability of the closed-loop nonlinear stochastic system based on fuzzy approximation and is characterized by asymptotic stability. Finally, a numerical simulation is offered to show the efficacy of the existing ADP-based control technique.

Topics & Concepts