An Efficient and Robust Aggregation Algorithm for Learning Federated CNN
Yan-Yang Lu, Lei Fan
Abstract
Federated learning is a privacy-protected way of decentralized machine learning. In a Federated Learning system, the server uses the aggregation algorithm to obtain a global model from clients. The traditional aggregation method called FedAvg is a simple arithmetic average, but it has never been proved efficient. Moreover, there are many potential attacks and network instability in Federated Learning systems. This paper aims to achieve two goals by replacing the server-side aggregation strategy: 1) Accelerate global model's convergence speed 2) Make global model reliable when facing network instability and offline attacks. Considering different clients' contributions have different impacts, we try to use gaussian distribution to weight clients' potential contributions. To make the aggregation process more modality to different neural network architecture, we try to solve the above problems on Convolution Neural Network as a representation. To work well on different functional units in neural networks, we also propose layer-wise optimizing steps. We have experimented on some representative tasks of Federated Learning, and the results show that our method exceeds FedAvg a lot on convergence speed. When simulating attack situations, our algorithm could be proved to maintain a reliable global model.