Deep Reinforcement Learning-Aided Bidding Strategies for Transactive Energy Market
Amirheckmat Taghizadeh, Mina Montazeri, Hamed Kebriaei
Abstract
The concept of transactive energy market (TEM) has been introduced to efficiently balance supply and demand across the electrical networks in a distributed manner. TEM allows consumers to sell/purchase a portion of their excess/lack of energy to/from other local consumers. In this article, a transactive framework is proposed for a distribution network in which transactive agents participate in a local market and submit their bids to local TEM for day-ahead planning. However, due to complexity of the market and effects of a multitude of factors in the market outcome, deriving the optimal bid is not a trivial task. In order to learn the optimal bidding strategy in such a complex system with incomplete information, soft actor-critic method is utilized. Two scenarios are analyzed and compared. The first scenario assumes only one agent uses the learning method, while in the second scenario a number of residents form a coalition with a representative learning agent. Then, allocation of power assigned to the whole coalition between its residents is discussed. Comprehensive simulations are conducted for the 37-bus IEEE distribution system in two different scenarios to compare nonlearner agents with learner agents in both scenarios.