Multi-agent deep reinforcement learning based demand response and energy management for heavy industries with discrete manufacturing systems
Atit Bashyal, Tina Boroukhian, Pakin Veerachanchai, Myanganbayar Naransukh, Hendro Wicaksono
Abstract
Energy-centric decarbonization of heavy industries, such as steel and cement, necessitates their participation in integrating Renewable Energy Sources (RES) and effective Demand Response (DR) programs. This situation has created the opportunities to research control algorithms in diverse DR scenarios. Further, the industrial sector’s unique challenges, including the diversity of operations and the need for uninterrupted production, bring unique challenges in designing and implementing control algorithms. Reinforcement learning (RL) methods are practical solutions to the unique challenges faced by the industrial sector. Nevertheless, research in RL for industrial demand response has not yet achieved the level of standardization seen in other areas of RL research, hindering broader progress. To propel the research progress, we propose a multi-agent reinforcement learning (MARL)-based energy management system designed to optimize energy consumption in energy-intensive industrial settings by leveraging dynamic pricing DR schemes. The study highlights the creation of a MARL environment and addresses these challenges by designing a general framework that allows researchers to replicate and implement MARL environments for industrial sectors. The proposed framework incorporates a Partially Observable Markov Decision Process (POMDP) to model energy consumption and production processes while introducing buffer storage constraints and a flexible reward function that balances production efficiency and cost reduction. The paper evaluates the framework through experimental validation within a steel powder manufacturing facility. The experimental results validate our framework and also demonstrate the effectiveness of the MARL-based energy management system. • Multi-agent reinforcement learning framework is developed for demand response in energy-intensive industries. • Centralized training with decentralized execution is proposed to optimize energy scheduling and coordination. • Attention mechanism is integrated to enhance cooperation efficiency and improve policy stability. • Proposed MARL-based strategy effectively balances energy cost reduction with production scheduling. • Adaptive and scalable energy management approaches are demonstrated through case study.