Compressed-Sensing-Based Practical and Efficient Privacy-Preserving Federated Learning

Abstract

Federated learning (FL) is a popular distributed learning framework that is proposed to address privacy concerns in traditional machine learning. However, recent research has highlighted an issue where model or gradient updates can be exploited to infer sensitive information from the training data, resulting in severe privacy leakage. Existing defenses against gradient leakage attacks often suffer from high computation overhead or compromised model performance. In addition, most defense methods lack sufficient protection for labels. To overcome the above shortcomings, we develop a compressed sensing (CS)-based practical and efficient privacy-preserving FL scheme. In order to provide simultaneous protection for both original data and labels, we propose a CS-based gradient perturbation method, which eliminates the information in the gradient that is commonly exploited by attackers to extract labels, and increases the discrepancy between the perturbed and original gradients. Meanwhile, double aggregation is adopted together to ensure individual gradients are not easily disclosed by attackers. We also design a novel gradient reconstruction method that adaptively estimates the true gradient sparsity used for decompressing, thereby improving the model performance in practical scenarios. Furthermore, our CS-based gradient compression reduces communication overhead and requires low computation overhead as it only involves fast matrix multiplication. Extensive experiment results demonstrate the strong privacy protection effects of our proposed scheme compared to other approaches across various settings, with advantages in terms of communication overhead, computation overhead, and model accuracy.

Topics & Concepts