Enhanced cyber-resilience in flexible energy markets for microgrids: A trust-aware federated deep reinforcement learning framework
Reza Sepehrzad, Hossein Hosseinalibeiki, Neda Taghinezhad, Nima Khosravi, Ahmed Al durra, Mahdieh S. Sadabadi
Abstract
Abstract This paper introduces a trust-aware, cyber-resilient approach to defend against adversarial cyberattacks against microgrid flexible energy markets. The new method applies federated day-ahead and real-time pricing controlled through a trust-aware federated deep reinforcement learning (TAFDRL) framework. The TAFDRL framework leverages encrypted updates, where each prosumer agent identifies a local soft actor-critic policy, along with a local policy that was updated based on Bayesian trust scores and tested for policy anomaly (latent policy anomaly model). Malicious updates are detected using the Mahalanobis distance from learned embedding space, which enables a trust-weighted average, which allows for secure model aggregation of individual learning. The TAFDRL framework was tested under numerous scenarios for false data injection (FDI), denial-of-service (DoS), and hybrid attack. The TAFDRL framework demonstrated a 94% cumulative reward rate (with 92% stability) and very high accuracy for detecting cyberattacks (< 2% error tolerance using 200 operational points), which maximized the reliable identification of suspicious policies that possess a persistent, resilient ability despite cyber threats. These results confirm that TAFDRL effectively balances cyber defense, aggressive learning performance, while maintaining economic stability. The research provides a robust framework for operating energy markets securely against an increasing number of cyber threats to energy distribution systems. • Trust-sensitive filtering and robust market model integrating demand response. • Secure model of day-ahead and real-time pricing against the cyber threats. • Attack-aware policy verification in future flexible energy markets of microgrid.