Optimal Secondary Control of Islanded AC Microgrids with Communication Time-delay Based on Multi-agent Deep Reinforcement Learning
Yang Xia, Yan Xu, Yu Wang, Suman Mondal, Souvik Dasgupta, Amit. K. Gupta
Abstract
In this paper, an optimal secondary control strategy is proposed for islanded AC microgrids considering communication time-delays.The proposed method is designed based on the data-driven principle, which consists of an offline training phase and online application phase.For offline training, each control agent is formulated by a deep neural network (DNN) and trained based on a multi-agent deep reinforcement learning (MA-DRL) framework.A deep deterministic policy gradient (DDPG) algorithm is improved and applied to search for an optimal policy of the secondary control, where a global cost function is developed to evaluate the overall control performance.In addition, the communication time-delay is introduced in the system to enrich training scenarios, which aims to solve the time-delay problem in the secondary control.For the online stage, each controller is deployed in a distributed way which only requires local and neighboring information for each DG.Based on this, the well-trained controllers can provide optimal solutions under load variations, and communication time-delays for online applications.Several case studies are conducted to validate the feasibility and stability of the proposed secondary control.