Defending Against Weight-Poisoning Backdoor Attacks for Parameter-Efficient Fine-Tuning
Shuai Zhao, Leilei Gan, Anh Tuan Luu, Jie Fu, Lingjuan Lyu, Meihuizi Jia, Jinming Wen
Abstract
Recently, various parameter-efficient finetuning (PEFT) strategies for application to language models have been proposed and successfully implemented.However, this raises the question of whether PEFT, which only updates a limited set of model parameters, constitutes security vulnerabilities when confronted with weight-poisoning backdoor attacks.In this study, we show that PEFT is more susceptible to weight-poisoning backdoor attacks compared to the full-parameter fine-tuning method, with pre-defined triggers remaining exploitable and pre-defined targets maintaining high confidence, even after finetuning.Motivated by this insight, we developed a Poisoned Sample Identification Module (PSIM) leveraging PEFT, which identifies poisoned samples through confidence, providing robust defense against weight-poisoning backdoor attacks.Specifically, we leverage PEFT to train the PSIM with randomly reset sample labels.During the inference process, extreme confidence serves as an indicator for poisoned samples, while others are clean.We conduct experiments on text classification tasks, five finetuning strategies, and three weight-poisoning backdoor attack methods.Experiments show near 100% success rates for weight-poisoning backdoor attacks when utilizing PEFT.Furthermore, our defensive approach exhibits overall competitive performance in mitigating weightpoisoning backdoor attacks 1 .