Novel Architecture of Energy Management Systems Based on Deep Reinforcement Learning in Microgrid
Seongwoo Lee, Joonho Seon, Young Ghyu Sun, Soo Hyun Kim, Chanuk Kyeong, Dong In Kim, Jin Young Kim
Abstract
In microgrids, energy management systems (EMS) have been considered essential systems to optimize energy scheduling, control and operation for reliable power systems. Conventional EMS researches have been predominantly performed by employing demand-side management and demand response (DR). Nonetheless, multi-action control in EMS is confronted with operational challenges in terms of the profitability and stability. In this paper, energy information systems (EIS), energy storage systems (ESS), energy trading risk management systems (ETRMS), and automatic DR (ADR) are integrated to efficiently manage the profitability and stability of the whole EMS by optimal energy scheduling. The proposed microgrid EMS architecture is optimized by using proximal policy optimization (PPO) algorithm, which has been known to have good performance in terms of learning stability and complexity. A novel performance metric, represented as a burden of load and generation (BoLG), is proposed to evaluate the energy management performance. The BoLG is incorporated into the reward settings for optimizing the management of multi-action controls such as load shifting, energy charging-discharging, and transactions. From the simulation results, it is confirmed that the proposed architecture can improve energy management performance with the proper trade-off between stability and profitability, compared to dynamic programming (DP)-based and double deep Q-network (DDQN)-based operation.