Orchestrating Smart Grid Demand Response Operations With URLLC and MuZero Learning
Mohammad Belayet Hossain, Shiva Raj Pokhrel, Jinho Choi
Abstract
Improving reliability and response time in decision-making is crucial for efficient demand response (DR) programs in smart grid (SG) environments. By precisely predicting the DR in near real time, consumer premises can be more prepared to optimize energy utilization. We propose and develop an ultrareliable low-latency communication (URLLC)-based machine learning paradigm for orchestrating DR with guaranteed reliability and timeliness. To understand the context in depth and develop new insights, we use a random forest (RF) algorithm to predict the DR program. After that, we employ MuZero reinforcement learning (MuZero RL) on top of RF-based learning with URLLC, which leverages an efficient learned model under such SG DR dynamics. It considerably improves decision-making delays with better generalization to unforeseen situations. In sharp contrast to the state-of-the-art approaches, we observe that MuZero RL enables continuous learning by self-play, achieves higher sample efficiency, and adapts well to the underlying dynamic environments. Such features of an intelligent agent are precious in the context of the considered DR program. We develop theoretical derivations and analyses to study and utilize the framework’s capabilities and demonstrate that URLLC can substantially reduce energy costs.