Analysis of Evolutionary Dynamics for Bidding Strategy Driven by Multi-Agent Reinforcement Learning
Ziqing Zhu, Ka Wing Chan, Siqi Bu, Siu Wing Or, Xiang Gao, Shiwei Xia
Abstract
In this letter, the evolutionary game theory (EGT) with replication dynamic equations (RDEs) is adopted to explicitly determine the factors affecting energy providers’ (EPs) willingness of using the market power to uplift the price in the bidding procedure, which could be simulated using the win-or-learn-fast policy hill climbing (WoLF-PHC) algorithm as a multi-agent reinforcement learning (MARL) method. Firstly, empirical and numerical connections between WoLF-PHC and RDEs is proved. Then, by formulating RDEs of the bidding procedure, three factors affecting the bidding strategy preference are revealed, including the load demand, severity of congestion, and the price cap. Finally, the impact of these factors on the converged bidding price is demonstrated in case studies, by simulating the bidding procedure driven by WoLF-PHC.