Continuous-Time Stochastic Policy Iteration of Adaptive Dynamic Programming
Qinglai Wei, Tianmin Zhou, Jingwei Lu, Yu Liu, Shuai Su, Jun Xiao
Abstract
In this article, we study the optimal control problem of continuous-time (CT) time-invariant nonlinear systems with stochastic nonlinear disturbances. A new stochastic adaptive dynamic programming (ADP) method is developed to solve the Hamilton–Jacobi–Bellman equation (HJBE). Under the conditional expectation, the value function and the control law are successively approximated simultaneously. The asymptotic stability of the closed-loop stochastic system in probability is analyzed by the stochastic Lyapunov direct method, and the convergence of the developed ADP method is given. Finally, four simulations illustrate the effectiveness of the developed method.
Topics & Concepts
Dynamic programmingBellman equationNonlinear systemStochastic programmingHamilton–Jacobi–Bellman equationStochastic controlMathematical optimizationLyapunov functionOptimal controlMathematicsConvergence (economics)Exponential stabilityConditional expectationStochastic optimizationInvariant (physics)Adaptive controlComputer scienceControl theory (sociology)Control (management)Mathematical physicsEconomicsEconomic growthEconometricsArtificial intelligencePhysicsQuantum mechanicsAdaptive Dynamic Programming ControlMechanical Circulatory Support Devices