摘要
The emergence of fine-tuning-as-a-service has revealed a new vulnerability in large language models (LLMs). A mere handful of malicious data uploaded by users can subtly manipulate the fine-tuning process, leading to a compromised alignment state. Existing methods to counteract fine-tuning attacks typically require substantial computational resources. Even with parameter-efficient techniques like LoRA, gradient updates remain essential. To address these challenges, we propose Neuron-Level Safety Realignment (NLSR), a training-free framework that restores the safety of LLMs based on the similarity difference of safety-critical neurons before and after fine-tuning. The core of our framework is first to construct a safety reference model from an initially aligned model to amplify safety-related features in neurons. We then utilize this reference model to identify safety-critical neurons, which we prepare as patches. Finally, we selectively restore only those neurons that exhibit significant similarity differences by transplanting these prepared patches, thereby minimally altering the fine-tuned model. Extensive experiments demonstrate significant safety enhancements in fine-tuned models across multiple downstream tasks, while greatly maintaining task-level accuracy. Our findings indicate that safety-critical neurons exhibit significant regional variations after fine-tuning, which can be effectively corrected through neuron transplantation from the reference model without the need for additional training.
| 源语言 | 英语 |
|---|---|
| 页(从-至) | 25706-25714 |
| 页数 | 9 |
| 期刊 | Proceedings of the AAAI Conference on Artificial Intelligence |
| 卷 | 39 |
| 期 | 24 |
| DOI | |
| 出版状态 | 已出版 - 11 4月 2025 |
| 活动 | 39th Annual AAAI Conference on Artificial Intelligence, AAAI 2025 - Philadelphia, 美国 期限: 25 2月 2025 → 4 3月 2025 |
学术指纹
探究 'NLSR: Neuron-Level Safety Realignment of Large Language Models Against Harmful Fine-Tuning' 的科研主题。它们共同构成独一无二的学术指纹。引用此
- APA
- Author
- BIBTEX
- Harvard
- Standard
- RIS
- Vancouver