FedDRL: Trustworthy Federated Learning Model Fusion Method Based on Staged Reinforcement Learning
Leiming Chen, Weishan Zhang, Cihao Dong, Ziling Huang, Yuming Nie, Zhaoxiang Hou, Sibo Qiao, Chee Wei Tan
Abstract
Federated learning facilitates collaborative data analysis among multiple participants while preserving user privacy. However, conventional federated learning approaches, typically employing weighted average techniques for model fusion, confront two significant challenges: 1. The inclusion of malicious models in the fusion process can drastically undermine the accuracy of the aggregated global model. 2. Due to the heterogeneity problem of devices and data, the number of client samples does not determine the weight value of the model. To solve those challenges, we propose a trustworthy model fusion method based on reinforcement learning (FedDRL), which includes two stages. In the first stage, we propose a reliable client selection mechanism to exclude malicious models from the fusion process. In the second stage, we propose an adaptive model fusion method that dynamically assigns weights based on model quality to aggregate the best global models. Finally, we validate our approach against five distinct model fusion scenarios, demonstrating that our algorithm significantly enhanced reliability without compromising accuracy.