Federated Learning Client Selection Mechanism Under System and Data Heterogeneity
Fa Xin, Jinghui Zhang, Junzhou Luo, Fang Dong
Abstract
Federated learning (FL) has been proposed to train a global model by distributed architecture, while keeping the training data local. Owing to the large scale of clients in FL, all clients to participate in training is not feasible. The heterogeneity of clients, including system and data heterogeneity, also poses huge challenge to the client selection problem. Traditional client selection mechanisms can’t handle these heterogeneities effectively, which lead to poor training efficiency. Hence, this paper comprehensively considers system and data heterogeneity to select clients and dynamically adjust the number of selected clients. For system heterogeneity, we build the latency model to predict the training time for selecting clients with best performance including CPU frequency, the size of dataset and transmission power. Besides, for data heterogeneity, the cluster model is established to cluster clients for alleviating the accuracy jitter owing to the non independent and identically distributed (Non-IID) dataset. We formulate the client selection problem aiming to minimize the overall training time on the premise of accuracy, and design the Federated Client Cluster and latency-Prediction Selection (FCCPS) algorithm to solve this problem. With extensive simulations, we show that the FCCPS algorithm can reduce the training time by up to 21% on Cifar-10 dataset and 13% on FashionMNIST dataset, as compared to FedAvg.