Scalable Decentralized Multi-Agent Federated Reinforcement Learning: Challenges and Advances
Praveen Kumar Myakala, Srikanth Kamatala
Abstract
The increasing prevalence of decentralized multiagent systems has spurred interest in Federated Reinforcement Learning (FRL) as a privacy-preserving framework for collaborative learning. However, scaling FRL to multi-agent settings introduces significant challenges, particularly in communication efficiency, decentralized aggregation, and handling nonstationary environments. This survey explores recent advancements in Scalable Decentralized Multi-Agent Federated Reinforcement Learning (MA-FRL), with a focus on communication efficient strategies and decentralized aggregation techniques. We review key approaches such as selective agent communication, local model updates, and gradient compression, analyzing their impact on scalability, convergence, and performance trade-offs. Additionally, we highlight comparative insights into different methods, their limitations, and real-world applicability in decentralized systems such as autonomous vehicles and smart grids. By identifying open challenges, including robustness against adversarial attacks and adaptive communication mechanisms, we outline promising directions for advancing decentralized MAFRL.