Federated Fine-Tuning of Large Language Models with Privacy Preservation and Cross-Domain Semantic Alignment
Sibo Wang, Song Han, Ziyu Cheng, Ming Wang, Jianping Li
Abstract
This paper presents a federated fine-tuning framework for large language models that addresses the challenges of multi-source heterogeneity and privacy sensitivity. The method incorporates a differential privacy perturbation strategy at the local client level to protect sensitive gradient information and prevent data leakage during cross-device collaboration. A domain adaptation module based on feature distribution alignment is introduced to reduce semantic shifts between source and target domains using maximum mean discrepancy optimization and attention-guided mechanisms. The overall architecture integrates local modeling with global parameter aggregation, forming a closed loop of federated alignment and global integration for efficient, secure, and cross-domain semantic modeling. The experimental design includes multidimensional sensitivity evaluations across privacy perturbation levels, label missingness, and domain distribution shifts. Results demonstrate that the proposed method achieves superior performance in key metrics such as Perplexity, MMD, and Domain Accuracy, confirming its effectiveness in jointly optimizing privacy protection and cross-domain generalization.