跳到主要导航 跳到搜索 跳到主要内容

NLSR: Neuron-Level Safety Realignment of Large Language Models Against Harmful Fine-Tuning

  • Xin Yi
  • , Shunfan Zheng
  • , Linlin Wang*
  • , Gerard de Melo
  • , Xiaoling Wang
  • , Liang He
  • *此作品的通讯作者
  • East China Normal University
  • University of Potsdam

科研成果: 期刊稿件 › 会议文章 › 同行评审

摘要

The emergence of fine-tuning-as-a-service has revealed a new vulnerability in large language models (LLMs). A mere handful of malicious data uploaded by users can subtly manipulate the fine-tuning process, leading to a compromised alignment state. Existing methods to counteract fine-tuning attacks typically require substantial computational resources. Even with parameter-efficient techniques like LoRA, gradient updates remain essential. To address these challenges, we propose Neuron-Level Safety Realignment (NLSR), a training-free framework that restores the safety of LLMs based on the similarity difference of safety-critical neurons before and after fine-tuning. The core of our framework is first to construct a safety reference model from an initially aligned model to amplify safety-related features in neurons. We then utilize this reference model to identify safety-critical neurons, which we prepare as patches. Finally, we selectively restore only those neurons that exhibit significant similarity differences by transplanting these prepared patches, thereby minimally altering the fine-tuned model. Extensive experiments demonstrate significant safety enhancements in fine-tuned models across multiple downstream tasks, while greatly maintaining task-level accuracy. Our findings indicate that safety-critical neurons exhibit significant regional variations after fine-tuning, which can be effectively corrected through neuron transplantation from the reference model without the need for additional training.

源语言英语
页(从-至)25706-25714
页数9
期刊Proceedings of the AAAI Conference on Artificial Intelligence
卷39
期24
DOI
出版状态已出版 - 11 4月 2025
活动39th Annual AAAI Conference on Artificial Intelligence, AAAI 2025 - Philadelphia, 美国
期限: 25 2月 2025 → 4 3月 2025

学术指纹

探究 'NLSR: Neuron-Level Safety Realignment of Large Language Models Against Harmful Fine-Tuning' 的科研主题。它们共同构成独一无二的学术指纹。

引用此