Distributed Optimal Consensus Problem of Input Constrained Nonlinear Discrete-Time MASs: A Mode-Free Reinforcement Learning Approach
Shuxing Xuan, Hongjing Liang, Shihao Huang, Tieshan Li, Jiayue Sun
Abstract
In this article, a model-free reinforcement learning (RL) approach is proposed for solving the optimal consensus control issue of nonlinear discrete-time multiagent systems with input constraint. To address the challenge of solving the coupled discrete Hamilton-Jacobi-Bellman (HJB) equation, a RL approach based on actor-critic framework is proposed for optimal consensus control. A well-defined cost function is designed, and the actor and critic networks are updated through online learning to obtain the optimal controllers. Furthermore, the actuator's performance is often limited due to physical constraints. To address such actuator constraints, a gradual transition control (GTC) method is proposed, and update-free and update-weak policies are introduced to further optimize network performance. Additionally, in real-world distributed systems, the actor-critic networks deployed in each agent rely on data from neighboring agents, which necessitates addressing the issue of distributed synchronization. To address this challenge, the synchronization blocking method is designed, which designs additional control signals for each agent to handle these issues. Finally, two simulations under different scenarios are presented to verify the effectiveness of the proposed approach.