FedQNN: A Computation–Communication-Efficient Federated Learning Framework for IoT With Low-Bitwidth Neural Network Quantization
Yu Ji, Lan Chen
Abstract
Federated learning (FL) allows participants to train deep learning models collaboratively without disclosing their data to the server or any other participants, providing excellent value in the field of privacy-sensitive IoT. However, this distributed training paradigm requires clients to perform intensive computation for many iterations, which may exceed the capability of a typical IoT terminal with limited processing power, storage capacity, and energy budget. Heavy communication between the server and clients may also result in intolerant bandwidth requirements and energy consumption for many IoT systems. In this article, we introduce the FedQNN, a computation–communication-efficient FL framework for IoT scenarios. It is the first work that integrates ultralow-bitwidth quantization into the FL environment, allowing clients to perform lightweight fix-point computation efficiently with less power. Furthermore, both upstream and downstream data are significantly compressed for more efficient communication using a combination of sparsification and quantization strategies. We performed extensive experiments on a variety of data sets and models while comparing with other frameworks, and the results demonstrate that the proposed method can save up to 90% of our clients’ computational energy, reduce model sizes by 30+ times, and significantly compress both communication bandwidth and transmitted data size while maintaining reasonable accuracy. The robustness against the non-independent and identically distributed (I.I.D.) condition is also validated.