Real-Time Sequential Security-Constrained Optimal Power Flow: A Hybrid Knowledge-Data-Driven Reinforcement Learning Approach
Zhongkai Yi, Xue Wang, Cheng Yang, Chao Yang, Mengyang Niu, Wotao Yin
Abstract
To confront the inaccuracy and imperfection of the environmental model, this article proposes a hybrid knowledge-data-driven reinforcement learning (KDD-RL) approach to solve the sequential optimal power flow problem during real-time operation. An improved soft actor-critic algorithm is proposed to train the control policy and formulate the sequential dispatch commands to the generators. To promote the safe exploration of the reinforcement learning algorithm, a hybrid knowledge-data-driven safety layer is developed to convert the unsafe actions into the safety region. Furthermore, a security-constrained linear projection model with an inactive constraint identification process is proposed to accelerate the computation efficiency of the safety layer. Numerical simulation results verify the superiority and scalability of the proposed approach in improving the decision-making efficiency and promoting the security operation of the power systems.