Machine Unlearning on MUSE (Df/Dr Sets)
58.4VerbMem (Df)Original LLM
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Original LLMModel=LLaMA-2-7B2025.09 | 58.4 | — | 63.9 | — | 55.2 | — | -99.8 | — | — | — | |
| Task VectorModel=LLaMA-2-7B, Method Group=Representation-Based Methods2025.09 | 56.3 | — | 63.7 | — | 54.6 | — | -99.8 | — | — | — | |
| MismatchModel=LLaMA-2-7B, Method Group=Representation-Based Methods2025.09 | 42.8 | — | 52.6 | — | 45.7 | — | -99.8 | — | — | — | |
| KLModel=LLaMA-2-7B, Method Group=Gradient-Based Methods2025.09 | 27.4 | — | 50.2 | — | 44.8 | — | -96.1 | — | — | — | |
| Retained LLMModel=LLaMA-2-7B2025.09 | 20.8 | — | 33.1 | — | 55 | — | 0 | — | — | — | |
| WHPModel=LLaMA-2-7B, Method Group=Representation-Based Methods2025.09 | 19.7 | — | 21.2 | — | 28.3 | — | 109.6 | — | — | — | |
| GDModel=LLaMA-2-7B, Method Group=Representation-Based Methods2025.09 | 4.9 | — | 27.5 | — | 6.7 | — | 109.4 | — | — | — | |
| FLAT (JS)Model=LLaMA-2-7B, Method Group=FLAT Methods2025.09 | 1.9 | — | 36.2 | — | 38.5 | — | 47.1 | — | — | — | |
| FLAT (TV)Model=LLaMA-2-7B, Method Group=FLAT Methods2025.09 | 1.7 | — | 13.6 | — | 31.8 | — | 45.4 | — | — | — | |
| FLAT (Pearson)Model=LLaMA-2-7B, Method Group=FLAT Methods2025.09 | 1.6 | — | 0 | — | 0.2 | — | 26.8 | — | — | — | |
| NPO-RTModel=LLaMA-2-7B, Method Group=Gradient-Based Methods2025.09 | 1.2 | — | 54.6 | — | 40.5 | — | 105.8 | — | — | — | |
| OURS (GD+Sine)Model=LLaMA-2-7B2025.09 | 0.8 | — | 5.2 | — | 42.1 | — | 8.3 | — | — | — | |
| GAModel=LLaMA-2-7B, Method Group=Gradient-Based Methods2025.09 | 0 | — | 0 | — | 0 | — | 17 | — | — | — | |
| NPOModel=LLaMA-2-7B, Method Group=Gradient-Based Methods2025.09 | 0 | — | 0 | — | 0 | — | 15 | — | — | — | |
| FLAT (KL)Model=LLaMA-2-7B, Method Group=FLAT Methods2025.09 | 0 | — | 0 | — | 0 | — | 58.9 | — | — | — | |
| Gold-standard retrained2026.04 | — | — | — | — | — | — | — | — | 0.56 | — | |
| GradAscentretain free=true, Time=1117s2026.04 | — | — | — | — | — | — | — | — | 0.477 | 0.533 | |
| GradDiffretain free=false, Time=2928s2026.04 | — | — | — | — | — | — | — | — | 0.436 | 0.521 | |
| MC-WIN-Uretain free=true, Time=392s2026.04 | — | — | — | — | — | — | — | — | 0.429 | 0.529 | |
| NPOretain free=false, Time=3192s2026.04 | — | — | — | — | — | — | — | — | 0.55 | 0.533 | |
| Original model2026.04 | — | — | — | — | — | — | — | — | 0.555 | — | |
| RMUretain free=false, Time=1897s2026.04 | — | — | — | — | — | — | — | — | 0.496 | 0.529 | |
| SimNPOretain free=false, Time=3066s2026.04 | — | — | — | — | — | — | — | — | 0.527 | 0.534 |