Entropy to Mitigate Non-IID Data Problem on Federated Learning for the Edge Intelligence Environment
Fernanda C. Orlandi, Julio César Santos dos Anjos, Valderi Reis Quietinho Leithardt, Juan F. De Paz, Cláudio F. R. Geyer
Abstract
Machine Learning (ML) algorithms process input data making it possible to recognize and extract patterns from a large data volume. Likewise, Internet of Things (IoT) devices provide knowledge in a Federated Learning (FL) environment, sharing parameters without compromising their raw data. However, FL suffers with non-independent and identically distributed (non-iid) data, which means it is heterogeneos data and has biased input data, such as in smartphone data sources. This bias causes low convergence for ML algorithms, high energy and bandwidth consumption. This work proposes a method that mitigates non-iid data through a FedAvg-BE algorithm that provides Federated Learning with the border entropy evaluation to select good input from a non-iid data environment. Extensive experiments were performed using publicly available datasets to train deep neural networks. The experiment result evaluation demonstrates that execution time saves up to 22% for the MNIST dataset and 26% for the CIFAR-10 dataset, with the proposed model in Federated Learning settings. The results demonstrate the feasibility of the proposed model to mitigate non-iid data impact.