Data-Based Optimal Microgrid Management for Energy Trading With Integral <i>Q</i>-Learning Scheme
Yongfeng Lv, Zhaolong Wu, Xiaowei Zhao
Abstract
This article proposes an integral <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${Q}$ </tex-math></inline-formula> -learning scheme to study the optimum management strategy of the battery for energy trading in the microgrid. The obtained optimal strategies are managed to minimize the cost of the microgrid, and simultaneously guarantee the better performance of the battery such that service life of battery is extended. First, the microgrid model is constructed, where renewable energy and profiled loads are considered. To satisfy the microgrid demand and the environmental energy consumption, an integral <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${Q}$ </tex-math></inline-formula> -learning scheme will be developed for the input policy of the microgrid energy system such that the model dynamics are avoided. Moreover, the minimized cost and the property of the battery are considered in the performance function. Rather than the traditional <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${Q}$ </tex-math></inline-formula> -learning iterative scheme, this article proposes a self-learning architecture for the integral- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${Q}$ </tex-math></inline-formula> learning scheme with two neural networks. The first network can learn the optimal <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${Q}$ </tex-math></inline-formula> -function. Another network is used to learn the optimal action such that the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${Q}$ </tex-math></inline-formula> -function is minimized and load demand can be satisfied. The network weights are updated with the gradient method and the stability analysis is presented. Finally, the experiment data is used to verify the proposed integral <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${Q}$ </tex-math></inline-formula> -learning scheme.