Machine Unlearning on MUSE News (Extraction Strength, KnowMem ROUGE)
-99.81PrivLeakBase
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Base2026.06 | -99.81 | — | — | — | |
| RMU2026.06 | -99.73 | — | — | — | |
| SATIMPBackbone=Llama-3.2-1B-Instruct, Evaluation Setup=CLEAN, m=2, kappa=0.012026.02 | -52.2 | 2.81 | 17.3 | 17.9 | |
| SATIMPBackbone=Llama-3.2-1B-Instruct, Evaluation Setup=MPU, m=2, kappa=0.012026.02 | -52.1 | 2.8 | 17 | 17.6 | |
| SIMNPOBackbone=Llama-3.2-1B-Instruct, Evaluation Setup=CLEAN, m=2, kappa=0.012026.02 | -52 | 2.82 | 15.5 | 18.1 | |
| SIMNPOBackbone=Llama-3.2-1B-Instruct, Evaluation Setup=MPU, m=2, kappa=0.012026.02 | -52 | 2.79 | 16.9 | 17.5 | |
| UNDIALBackbone=Llama-3.2-1B-Instruct, Evaluation Setup=CLEAN, m=2, kappa=0.012026.02 | -49 | 1.6 | 13.6 | 21 | |
| UNDIALBackbone=Llama-3.2-1B-Instruct, Evaluation Setup=MPU, m=2, kappa=0.012026.02 | -49 | 1.6 | 13.7 | 20.7 | |
| NPOBackbone=Llama-3.2-1B-Instruct, Evaluation Setup=CLEAN, m=2, kappa=0.012026.02 | -28 | 2.79 | 16.3 | 17.8 | |
| NPOBackbone=Llama-3.2-1B-Instruct, Evaluation Setup=MPU, m=2, kappa=0.012026.02 | -27.5 | 2.8 | 16.7 | 17.8 | |
| Retrain2026.06 | -4.72 | — | — | — | |
| GA2026.06 | 5.22 | — | — | — | |
| NPO2026.06 | 14.99 | — | — | — | |
| GRADASCENTBackbone=Llama-3.2-1B-Instruct, Evaluation Setup=MPU, m=2, kappa=0.012026.02 | 19.2 | 0.975 | 0 | 0 | |
| GRADASCENTBackbone=Llama-3.2-1B-Instruct, Evaluation Setup=CLEAN, m=2, kappa=0.012026.02 | 19.3 | 0.975 | 0 | 0 | |
| SimNPO2026.06 | 35.26 | — | — | — | |
| MASC2026.06 | 41.99 | — | — | — | |
| GRADDIFFBackbone=Llama-3.2-1B-Instruct, Evaluation Setup=CLEAN, m=2, kappa=0.012026.02 | 79.7 | 0.985 | 3.81 | 2.92 | |
| GRADDIFFBackbone=Llama-3.2-1B-Instruct, Evaluation Setup=MPU, m=2, kappa=0.012026.02 | 82.9 | 1 | 3.46 | 1.23 | |
| NPO+KLR2026.06 | 87.03 | — | — | — | |
| GradDiff2026.06 | 105.16 | — | — | — |