A Novel Server-side Aggregation Strategy for Federated Learning in Non-IID situations
Jianhang Xiao, Chunhui Du, Zijing Duan, Wei Guo
Abstract
Federated learning has been a promising distributed machine learning approach in many fields like e-economic, autodriving and medical imaging for its privacy-aware manner. However, researchers have discovered that the performance of traditional federated learning approaches such as Federated Averaging (FEDAVG) declines extremely under Non-Independent and Identical (Non-IID) situations. We observed that part of the reason is the improper way of traditional federated learning’s server-side aggregation method.The contributions of clients in federated learning can be distinguished by their trained models’ validated accuracies. Based on that observation, we proposed a new federated learning algorithm, Accuracy Based Averaging (ABAVG), which improves the server-side aggregation method of traditional federated learning so that it can accelerate the convergence speed of federated learning in Non-IID situations. We extensively evaluate our proposed algorithm with FEDAVG as a baseline and we experiment on various Non-IID conditions to demonstrate the robust of our proposed algorithm. Experimental results show that the convergence speed averagely increased by 47% in Mnist dataset, 59% in Fashion-Mnist dataset and 33% in CIFAR-10 dataset in different data distributions by ABAVG.