Agent-Based Modeling in Electricity Market Using Deep Deterministic Policy Gradient Algorithm
Yanchang Liang, Chunlin Guo, Zhaohao Ding, Huichun Hua
Abstract
Game theoretic methods and simulations based on reinforcement learning (RL) are often used to analyze electricity market equilibrium. However, the former is limited to a simple market environment with complete information, and difficult to visually reflect the tacit collusion; while the conventional RL algorithm is limited to low-dimensional discrete state and action spaces, and the convergence is unstable. To address the aforementioned problems, this paper adopts deep deterministic policy gradient (DDPG) algorithm to model the bidding strategies of generation companies (GenCos). Simulation experiments, including different settings of GenCo, load and network, demonstrate that the proposed method is more accurate than conventional RL algorithm, and can converge to the Nash equilibrium of complete information even in the incomplete information environment. Moreover, the proposed method can intuitively reflect the different tacit collusion level by quantitatively adjusting GenCos' patience parameter, which can be an effective means to analyze market strategies.