A Novel Automatic Generation Control Method Based on the Large-Scale Electric Vehicles and Wind Power Integration Into the Grid
Lei Xi, Haokai Li, Jizhong Zhu, Yanying Li, Shouxiang Wang
Abstract
In order to solve the problem of frequency instability of power system due to strong random disturbance caused by large-scale electric vehicles and wind power grid connection, an improved reinforcement learning algorithm, namely, optimistic initialized double Q, is proposed in this article from the perspective of automatic generation control. The proposed algorithm uses the optimistic initialization principle to expand the agent action exploration space, so as to prevent Q-learning from falling into local optimum by greedy strategy; meanwhile, it integrates double Q-learning to solve the problem of overestimation of action value in traditional reinforcement learning based on Q-learning. In the algorithm, the hyperparameter <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\alpha_{\tau}$</tex-math> </inline-formula> is introduced to improve the learning efficiency, and the reward <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$b_{\tau}$</tex-math> </inline-formula> based on exploration times is introduced to increase the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$</tex-math> </inline-formula> value estimation to drive the exploration of the algorithm, so as to obtain the optimal solution. By simulating the two-area load frequency control model integrated with large-scale electric vehicles and the four-area interconnected power grid model integrated with large-scale wind power generation, it is verified that the proposed algorithm can obtain the global optimal solution, thus effectively solvinng the frequency instability caused by strong random disturbance in the grid-connected mode of large-scale wind power generation, and compared with many reinforcement learning algorithms, the proposed algorithm has better control performance.