Machine Unlearning on TOFU (10%)
1Forget Quality (FQ)Retain LLM
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Retain LLM2024.06 | 1 | 39.8 | 0.62 | 98.2 | — | |
| Retrained (Oracle)Backbone=LLaMA-2-7B-Chat2026.01 | 1 | — | 0.62 | — | — | |
| Retain2026.04 | 1 | — | — | — | -0.016 | |
| RetrainCondition=Best-epoch performance, Evaluation=Averaged over five seeds2025.10 | 1 | — | 0.62 | — | — | |
| Retrain2025.10 | 1 | — | 0.62 | — | — | |
| Retrain LLM2025.10 | 1 | — | 0.62 | — | — | |
| KIFBackbone=LLaMA-2-7B-Chat2026.01 | 0.99 | — | 0.62 | — | — | |
| ECOvariant=Zero-Out2026.04 | 0.9674 | — | — | — | -0.0014 | |
| DiPOCondition=Best-epoch performance, Evaluation=Averaged over five seeds2025.10 | 0.86 | — | 0.57 | — | — | |
| DiPO2025.10 | 0.86 | — | 0.57 | — | — | |
| DiPO2025.10 | 0.84 | — | 0.56 | — | — | |
| RELOAD2026.04 | 0.7 | — | — | — | -0.3384 | |
| ECOvariant=Rand Noise2026.04 | 0.5812 | — | — | — | -0.0028 | |
| AltPO2025.10 | 0.58 | — | 0.56 | — | — | |
| ULD2024.06 | 0.48 | 42.6 | 0.62 | 85.9 | — | |
| ULD2025.10 | 0.48 | — | 0.62 | — | — | |
| SimNPOBackbone=LLaMA-2-7B-Chat2026.01 | 0.45 | — | 0.62 | — | — | |
| NPO+GDCondition=Best-epoch performance, Evaluation=Averaged over five seeds2025.10 | 0.45 | — | 0.55 | — | — | |
| NPO+GDretain_loss=GD2024.06 | 0.29 | 25.7 | 0.53 | 41.1 | — | |
| NPOBackbone=LLaMA-2-7B-Chat2026.01 | 0.29 | — | 0.55 | — | — | |
| KL Min2026.04 | 0.181 | — | — | — | -0.6257 | |
| NPO+GD2025.10 | 0.17 | — | 0.53 | — | — | |
| NPOCondition=Best-epoch performance, Evaluation=Averaged over five seeds2025.10 | 0.1 | — | 0.07 | — | — | |
| NPO2024.06 | 0.09 | 15.2 | 0.26 | 15.3 | — | |
| NPO-RT2026.04 | 0.0783 | — | — | — | -0.126 | |
| NPO+KLretain_loss=KL2024.06 | 0.07 | 18.1 | 0.32 | 22.9 | — | |
| NPO-KL2026.04 | 0.0158 | — | — | — | -0.2634 | |
| NPO2026.04 | 0.0126 | — | — | — | -0.4556 | |
| GA+GDretain_loss=GD2024.06 | 0.009 | 19.6 | 0.17 | 23.9 | — | |
| GA+GD2025.10 | 0.009 | — | 0.51 | — | — | |
| NPO2025.10 | 0.0005 | — | 0 | — | — | |
| GA+KLretain_loss=KL2024.06 | 0.0002 | 12.1 | 0.05 | 18.6 | — | |
| Offset-NPO+KLuse_offset_objective=true2024.06 | 0 | 34.2 | 0.48 | 34.8 | — | |
| GA+KLCondition=Best-epoch performance, Evaluation=Averaged over five seeds2025.10 | 0 | — | 0.33 | — | — | |
| GA+KL2025.10 | 0 | — | 0.55 | — | — | |
| GACondition=Best-epoch performance, Evaluation=Averaged over five seeds2025.10 | 0 | — | 0 | — | — | |
| GA+GDCondition=Best-epoch performance, Evaluation=Averaged over five seeds2025.10 | 0 | — | 0.48 | — | — | |
| Offset-GA+KLuse_offset_objective=true2024.06 | 0 | 3.1 | 0.04 | 2.9 | — | |
| DPO2024.06 | 0 | 0.7 | 0 | 0.72 | — | |
| DPO+GDCondition=Best-epoch performance, Evaluation=Averaged over five seeds2025.10 | 0 | — | 0.05 | — | — | |
| DPO+GD2025.10 | 0 | — | 0 | — | — | |
| DPO+KLretain_loss=KL2024.06 | 0 | 0.7 | 0.03 | 0.81 | — | |
| Offset-DPO+KLuse_offset_objective=true2024.06 | 0 | 1.3 | 0.02 | 1.4 | — | |
| GA2024.06 | 0 | 0 | 0 | 0 | — | |
| DPO+GDretain_loss=GD2024.06 | 0 | 0.8 | 0 | 0.89 | — | |
| GA2025.10 | 0 | — | 0 | — | — | |
| IDK (rejection)Backbone=LLaMA-2-7B-Chat2026.01 | 0 | — | 0.54 | — | — | |
| GradDiffBackbone=LLaMA-2-7B-Chat2026.01 | 0 | — | 0.54 | — | — | |
| Gradient AscentBackbone=LLaMA-2-7B-Chat2026.01 | 0 | — | 0 | — | — | |
| Target LLM2024.06 | 0 | 98.6 | 0.62 | 98.2 | — | |
| OriginalCondition=Best-epoch performance, Evaluation=Averaged over five seeds2025.10 | 0 | — | 0.62 | — | — | |
| Original2025.10 | 0 | — | 0.62 | — | — | |
| Original LLM2025.10 | 0 | — | 0.62 | — | — | |
| OriginalBackbone=LLaMA-2-7B-Chat2026.01 | 0 | — | 0.62 | — | — | |
| Original2026.04 | 0 | — | — | — | 0 | |
| Grad Ascent2026.04 | 0 | — | — | — | -0.6257 | |
| Grad Diff2026.04 | 0 | — | — | — | -0.0434 | |
| Pref Opt2026.04 | 0 | — | — | — | -0.0862 | |
| Prompt2026.04 | 0 | — | — | — | -0.138 | |
| ECOvariant=Sign-Flip2026.04 | 0 | — | — | — | -0.0022 |