Unlearning on MUSE-Books Harry Potter (forget set 500 samples)
39.99R-Forget-500Base Model (Llama3.2-3B)
Evaluation results
| Method | Links | |
|---|---|---|
| Base Model (Llama3.2-3B)Backbone=Llama 3.2-3B-Instruct, Training Data Format=Raw book content2026.01 | 39.99 | |
| GA + KL (Dr)Backbone=Llama 3.2-3B-Instruct, Training Data Format=Raw book content, Regularization=KL-divergence2026.01 | 38.29 | |
| Refusal-TrainingBackbone=Llama 3.2-3B-Instruct, Training Data Format=DQR_f ∪ Dr2026.01 | 37.75 | |
| GA (DQA_f) + KL (Dr)Backbone=Llama 3.2-3B-Instruct, Training Data Format=QA samples, Regularization=KL-divergence2026.01 | 36.87 | |
| NPO (DQA_f)Backbone=Llama 3.2-3B-Instruct, Training Data Format=QA samples2026.01 | 34.28 | |
| NPO + KL (Dr)Backbone=Llama 3.2-3B-Instruct, Training Data Format=Raw book content, Regularization=KL-divergence2026.01 | 33.62 | |
| NPOBackbone=Llama 3.2-3B-Instruct, Training Data Format=Raw book content2026.01 | 26.83 | |
| NPO (DQA_f) + KL (Dr)Backbone=Llama 3.2-3B-Instruct, Training Data Format=QA samples, Regularization=KL-divergence2026.01 | 25.6 | |
| SimNPOBackbone=Llama 3.2-3B-Instruct, Training Data Format=Raw book content2026.01 | 21.41 | |
| DUETBackbone=Llama 3.2-3B-Instruct, Training Data Format=Dquery_f ∪ Dr2026.01 | 5.98 | |
| FLATBackbone=Llama 3.2-3B-Instruct, Training Data Format=Raw book content2026.01 | 0.64 | |
| GABackbone=Llama 3.2-3B-Instruct, Training Data Format=Raw book content2026.01 | 0 | |
| GA (DQA_f)Backbone=Llama 3.2-3B-Instruct, Training Data Format=QA samples2026.01 | 0 |