Federated Analytics Informed Distributed Industrial IoT Learning With Non-IID Data
Zibo Wang, Yifei Zhu, Dan Wang, Zhu Han
Abstract
The increasing concerns of communication overheads and data privacy greatly challenge the gather-and-analyze paradigm of data-driven tasks currently adopted by the industrial IoT deployments. The federated paradigm resolves this challenge by performing tasks collaboratively without uploading the raw data. However, the inherent data heterogeneity (skewness) of diverse industrial IoT data holders significantly degrades the performances of all kinds of federated industrial IoT learning tasks. Quantifying this skewness is non-trivial and cannot be solved by the existing federated learning techniques. In this paper, we propose a Federated skewness Analytics and Client Selection mechanism (FedACS) to quantify the data skewness in a privacy preserving way and use this information to help downstream federated learning tasks. FedACS provably estimates the skewness of the clients using the Hoeffding's inequality based on the distilled insights of edge data in the form of gradient. It then gracefully handles the drifting estimation and robustly selects clients with milder skewness using a novel dueling bandit approach. FedACS gains advantages in privacy preservation, infrastructure reuse, and optimized overheads. Extensive experiments on open datasets demonstrate that FedACS reduces the accuracy degradation by <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\sim$</tex-math></inline-formula> 78.2%, and accelerates the FL convergence for <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\sim 2.4\times$</tex-math></inline-formula> .