Deep reinforcement learning as a tool for the analysis and optimization of energy flows in multi-energy systems
Alberto Franzoso, Gabriele Fambri, Marco Badami
Abstract
Deep Reinforcement Learning algorithms not only facilitate the development of optimized control strategies but also serve as powerful tools to explore complex problems and uncover non-obvious control solutions. This paper investigates the application of Deep Reinforcement Learning to optimize a Multi-Energy System in the presence of high Renewable Energy Source penetration. Key energy conversion technologies, such as Combined Heat and Power, Battery Energy Storage Systems, Heat Pumps, and Power-to-Gas, enable bidirectional energy exchanges across different networks, thereby fostering operational synergies. Since these interconnections create interdependencies in which energy flows within one sector significantly affect those in another, the complexity of optimization increases. The aim of this study has been to demonstrate the benefits of a method that can be used to interpret strategies implemented by a Deep Reinforcement Learning algorithm, thereby ultimately increasing the possibility of making optimal decisions. This approach has led to the creation of an optimized rule-based mechanism which has been used to analyze the Multi-Energy System, identify the most advantageous technologies (heat pumps, electric batteries and power-togas, respectively), and highlight the importance of implementing an optimized strategy to achieve effective energy management. Such an optimized strategy led to a reduction in natural gas consumption of about 15%, a decrease in CO 2 emissions of 18%, and a reduction in fuel and electricity costs of 17%.