Multiagent Soft Actor–Critic Learning for Distributed ESS Enabled Robust Voltage Regulation of Active Distribution Grids
Yongdong Chen, Youbo Liu, Hang Yin, Zhiyuan Tang, Gao Qiu, Junyong Liu
Abstract
In this article, a novel data-driven robust voltage regulation method employing the multiagent soft actor–critic algorithm for photovoltaic-rich distribution grids considering storage lifetime and topology flexibility is proposed. In the proposed scheme, the active and reactive power from distributed energy storage system (ESS) are coordinated to deliver effective voltage support. To account for the long-term influence of ESS behavior on its lifetime, the life costs associated with the energy throughput are firstly formulated into the reward function of the Markov game-based voltage regulation model. Then, the topology status is represented by continuous variables transformed via Gumbel-softmax and embedded into the local observation of ESS agents for being aware of topology variations due to operational reconfiguration. In addition, to enhance the robustness of the voltage regulation method against imperfect measurements, the designed state space incorporates solely partially observed information from the entire distribution networks. Numerical simulations on IEEE 69-bus and IEEE 141-bus test systems confirm the outperforming of the proposed method over the previously implemented voltage regulation approaches.