Split Aggregation: Lightweight Privacy-Preserving Federated Learning Resistant to Byzantine Attacks
Zhi Lu, Songfeng Lu, Yongquan Cui, Xueming Tang, Junjun Wu
Abstract
Federated Learning (FL), a distributed learning paradigm optimizing communication costs and enhancing privacy by uploading gradients instead of raw data, now confronts security challenges. It is particularly vulnerable to Byzantine poisoning attacks and potential privacy breaches via inference attacks. While homomorphic encryption and secure multi-party computation have been employed to design robust FL mechanisms, these predominantly rely on Euclidean distance or median-based metrics and often fall short in comprehensively defending against advanced poisoning attacks, such as adaptive attacks. Addressing this issue, our study introduces “Split-Aggregation", a lightweight privacy-preserving FL solution capable of withstanding adaptive attacks. This method maintains a computational complexity of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i> ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">dkN</i> + <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i> <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> ) and a communication overhead of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i> ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">dN</i> ), performing comparably to FedAvg when <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i> = 10. Here, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">d</i> represents the gradient dimension, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</i> the number of users, and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i> the rank chosen during random singular value decomposition. Additionally, we utilize adaptive weight coefficients to mitigate gradient descent issues in honest users caused by non-independent and identically distributed (Non-IID) data. The proposed method’s security and robustness are theoretically proven, with its complexity thoroughly analyzed. Experimental results demonstrate that at <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i> = 10, this method surpasses the top-1 accuracy of current state-of-the-art robust privacy-preserving FL approaches. Moreover, opting for a smaller <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i> significantly boosts efficiency with only marginal compromises in accuracy.