Supporting virtual power plants decision-making in complex urban environments using reinforcement learning
Chengyang Liu, Rebecca Yang, Xinghuo Yu, Qian Sun, Gary Rosengarten, Ariel Liebman, Ron Wakefield, Peter S. P. Wong, Kaige Wang
Abstract
Virtual Power Plants (VPPs) are becoming popular for managing energy supply in urban environments with Distributed Energy Resources (DERs). However, decision-making for VPPs in such complex environments is challenging due to multiple uncertainties and complexities. This paper proposes an approach that optimizes decision-making for VPPs using Reinforcement Learning (RL) in urban environments with diverse supply-demand profiles and DERs. The approach addresses challenges related to integrating renewable energy sources and achieving energy efficiency. An RL-based VPP system is trained and tested under different scenarios, and a case study is conducted in a real-world urban environment. The proposed approach achieves multi-objective optimization by performing actions such as load-shifting, demand offsetting, and providing ancillary services in response to demand, renewable generators, and market signals. The study validates the effectiveness and robustness of the proposed approach under complex environmental conditions. Results demonstrate that the approach provides optimized decisions in various urban environments with different available resources and supply-demand profiles. This paper contributes to understanding the use of RL in optimizing VPP decision-making and provides valuable insights for policymakers and practitioners in sustainable and resilient cities.