A Safe Policy Learning-Based Method for Decentralized and Economic Frequency Control in Isolated Networked-Microgrid Systems
Yang Xia, Yan Xu, Yu Wang, Suman Mondal, Souvik Dasgupta, Amit Kumar Gupta, Gaurav M. Gupta
Abstract
Multiple microgrids can be interconnected to form a networked-microgrid (NMG) system. In this paper, a data-driven decentralized economic frequency control method is proposed for isolated NMG systems. Based on multi-agent deep reinforcement learning (MA-DRL) framework, each DRL agent controls the generator and energy storage system (ESS) in each microgrid of the NMG. For offline training stage, the optimal control strategy is learned by soft actor-critic (SAC) algorithm with a global reward, which aims to restore system frequency while considering economy. Besides, to satisfy system constraints and avoid high learning costs, a safety model scheme is designed and trained to support each DRL agent. Since the agents are trained in the centralized learning process at offline stage, they are able to coordinate in a decentralized manner for online application, which only requires local information to generate optimal control actions. Also, the trained safety model can be applied for online stage to monitor and guide online actions. Finally, numerical tests are conducted to demonstrate the feasibility and effectiveness of the proposed method under the variation of renewable generation and load demand.