Multi-Agent Proximal Policy Optimization-Based Dynamic Client Selection for Federated AI in 6G-Oriented Internet of Vehicles
Tianqi Yu, Xianbin Wang, Jianling Hu, Jianfeng Yang
Abstract
In the era of 5G-Advanced and 6 G, decentralized machine learning algorithms, particular federated learning (FL), has become a critical enabling technology for future Internet of Vehicles (IoV) systems in achieving effective edge intelligence and collaboration. However, due to the dynamic IoV environments and varying resource conditions, not all intelligent connected vehicles (ICVs) can effectively participate in the client-server based FL framework. Thus, dynamic vehicle client selection becomes essential for federated AI in IoV to achieve high model accuracy and low system overhead. To overcome the client selection challenge due to resource heterogeneity and vehicle mobility, a multi-agent proximal policy optimization (MAPPO)-based dynamic client selection mechanism has been proposed in this paper. Specifically, an IoV system is modeled as a multi-agent system (MAS), where the ICVs are regarded as agents. The optimization problem of client selection is cast to a decentralized partial observation Markov decision process (DEC-POMDP). Subsequently, a policy-gradient multi-agent deep reinforcement learning (MADRL) algorithm, MAPPO, is developed to resolve the DEC-POMDP problem. Simulation results indicate that the proposed MAPPO-based federated client selection mechanism optimizes the system overhead and model accuracy of federated model training compared to the benchmark mechanisms.