Unlearning on MUSE-Books Harry Potter (100 samples, forget set)
32.13R-ForgetBase Model (Llama3.2-3B)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Base Model (Llama3.2-3B)Backbone=Llama 3.2-3B-Instruct, Training Data Format=Raw book content, Regularization=None2026.01 | 32.13 | 0 | |
| Refusal-TrainingBackbone=Llama 3.2-3B-Instruct, Training Data Format=DQR_f ∪ Dr, Regularization=None2026.01 | 31.02 | -6.6 | |
| NPO (DQA_f)Backbone=Llama 3.2-3B-Instruct, Training Data Format=QA samples, Regularization=None2026.01 | 30.19 | -31.42 | |
| NPO + KL (Dr)Backbone=Llama 3.2-3B-Instruct, Training Data Format=Raw book content, Regularization=KL-divergence2026.01 | 28.92 | 3.58 | |
| GA (DQA_f) + KL (Dr)Backbone=Llama 3.2-3B-Instruct, Training Data Format=QA samples, Regularization=KL-divergence2026.01 | 27.44 | 7.63 | |
| GA + KL (Dr)Backbone=Llama 3.2-3B-Instruct, Training Data Format=Raw book content, Regularization=KL-divergence2026.01 | 27.2 | -0.27 | |
| NPOBackbone=Llama 3.2-3B-Instruct, Training Data Format=Raw book content, Regularization=None2026.01 | 24.18 | -0.16 | |
| NPO (DQA_f) + KL (Dr)Backbone=Llama 3.2-3B-Instruct, Training Data Format=QA samples, Regularization=KL-divergence2026.01 | 21.55 | -33.85 | |
| SimNPOBackbone=Llama 3.2-3B-Instruct, Training Data Format=Raw book content, Regularization=None2026.01 | 17.6 | -9.15 | |
| DUETBackbone=Llama 3.2-3B-Instruct, Training Data Format=Dquery_f ∪ Dr, Regularization=None2026.01 | 4.27 | 55.9 | |
| FLATBackbone=Llama 3.2-3B-Instruct, Training Data Format=Raw book content, Regularization=None2026.01 | 0.47 | 42.51 | |
| GABackbone=Llama 3.2-3B-Instruct, Training Data Format=Raw book content, Regularization=None2026.01 | 0 | -48.76 | |
| GA (DQA_f)Backbone=Llama 3.2-3B-Instruct, Training Data Format=QA samples, Regularization=None2026.01 | 0 | 38.62 |