Reinforcement Learning-Based Energy Trading and Management of Regional Interconnected Microgrids
Shuai Liu, Siyuan Han, Shanying Zhu
Abstract
In this paper, we present a Value-Decomposition Deep Deterministic Policy Gradients (V3DPG) based Reinforcement Learning (RL) method for energy trading and management of regional interconnected microgrids (MGs). In practice, the state of an MG is time-varying and the traded energy flows continuously, which is generally neglected in researches. To address this problem, an Actor-Critic framework is adopted. Each MG has to make energy trading decision based on local observation and has no access to any knowledge of other MGs. We bring in the idea of value-decomposition in the training process to ensure the generation of feasible cooperative policies while maintaining MGs’ privacy and autonomous decision-making ability. Furthermore, in light of the uncertainty and fluctuation of renewable energy generation and users’ demand, a recurrent neural network (RNN) with Burn-In initialization is combined with critic network to achieve implicit predictions. Meanwhile, we also take Energy Storage System (ESS) with operational constraints into consideration and deem it as a virtual market innovatively. Experiments have been carried out under real-world data to verify the merit of the proposed method, compared to existing RL-based works.