# Erase or Hide? Suppressing Spurious Unlearning Neurons for Robust Unlearning

Source: [Erase or Hide? Suppressing Spurious Unlearning Neurons for Robust Unlearning](<https://www.sota2.com/research/paper/68fa7baa1e209131cea89c3d>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-06T05:00:05.533Z

Authors: Nakyeong Yang, Dong-Kyum Kim, Jea Kwon, Minsung Kim, Kyomin Jung, Meeyoung Cha

Published: 2025-09-26T12:27:23.000Z

## Abstract

Large language models trained on web-scale data can memorize private or sensitive knowledge, raising significant privacy risks. Although some unlearning methods mitigate these risks, they remain vulnerable to "relearning" during subsequent training, allowing a substantial portion of forgotten knowledge to resurface. In this paper, we show that widely used unlearning methods cause shallow alignment: instead of faithfully erasing target knowledge, they generate spurious unlearning neurons that amplify negative influence to hide it. To overcome this limitation, we introduce Ssiuu, a new class of unlearning methods that employs attribution-guided regularization to prevent spurious negative influence and faithfully remove target knowledge. Experimental results confirm that our method reliably erases target knowledge and outperforms strong baselines across two practical retraining scenarios: (1) adversarial injection of private data, and (2) benign attack using an instruction-following benchmark. Our findings highlight the necessity of robust and faithful unlearning methods for safe deployment of language models.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Machine Unlearning on TOFU (Attack Robustness and Forgetting)](<https://www.sota2.com/research/sota/machine-unlearning-on-tofu-attack-robustness-and-forgetting.md>): Machine Unlearning; TOFU; Forgetting Score (FS): 17.75.

- [Knowledge Unlearning on FaithUn](<https://www.sota2.com/research/sota/knowledge-unlearning-on-faithun.md>): Knowledge Unlearning; FaithUn; FS: 0.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2509.22263>)

[PDF](<https://arxiv.org/pdf/2509.22263.pdf>)