${\sf FedRLChain}$: Secure Federated Deep Reinforcement Learning With Blockchain
Sujit Chowdhury, Arnab Mukherjee, Raju Halder
Abstract
This article introduces <inline-formula><tex-math notation="LaTeX">${\sf FedRLChain}$</tex-math></inline-formula> , a novel framework for blockchain-based secure federated deep reinforcement learning, which allows users to securely and collaboratively train a Deep Reinforcement Learning (DRL) model by plugging appropriate aggregation and verification algorithms for specific problems. Unlike existing systems, <inline-formula><tex-math notation="LaTeX">${\sf FedRLChain}$</tex-math></inline-formula> adopts 1) a novel verification algorithm to prevent malicious clients, 2) an aggregation weight scheme from preventing the global model from getting biased toward any client, and 3) a variant of traditional FedAverage algorithm to accelerate the convergence process. We perform a rigorous experimental evaluation of <inline-formula><tex-math notation="LaTeX">${\sf FedRLChain}$</tex-math></inline-formula> considering the classic cart-pole problem, and we show a significant improvement in the number of epochs and time required for model convergence w.r.t. the state-of-the-art frameworks – DDQL, BAFFLE, and BASE-PIoT.