Machine Unlearning on TOFU (Comprehensive Metrics)
89Forget QualityULD
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ULDtraining=adapters only2025.10 | 89 | — | 29 | 47 | — | — | — | — | — | — | — | |
| IHLtraining=adapters only2025.10 | 68 | — | 25 | 62 | — | — | — | — | — | — | — | |
| NPOtraining=adapters only2025.10 | 58 | — | 14 | 97 | — | — | — | — | — | — | — | |
| NPO + CEtraining=adapters only2025.10 | 53 | — | 16 | 93 | — | — | — | — | — | — | — | |
| NPO + KLtraining=adapters only2025.10 | 51 | — | 26 | 90 | — | — | — | — | — | — | — | |
| IHL + CEtraining=adapters only2025.10 | 51 | — | 52 | 45 | — | — | — | — | — | — | — | |
| IHL + KLtraining=adapters only2025.10 | 42 | — | 47 | 48 | — | — | — | — | — | — | — | |
| TOFUBaseline=GA+KL2026.04 | 40.46 | 16.74 | 26.8 | 47.34 | 47.72 | 18.76 | 11.08 | 54.53 | — | — | — | |
| GA + CEtraining=adapters only2025.10 | 34 | — | 15 | 97 | — | — | — | — | — | — | — | |
| TOFUBaseline=GA2026.04 | 26.57 | 29.95 | 36.32 | 61.8 | 41.73 | 58.67 | 57.41 | 52.22 | — | — | — | |
| TOFUBaseline=GA+GD2026.04 | 26.57 | 29.12 | 31.54 | 70.63 | 33.42 | 54.69 | 49.62 | 45.92 | — | — | — | |
| GAtraining=adapters only2025.10 | 23 | — | 8 | 98 | — | — | — | — | — | — | — | |
| AStraining=adapters only2025.10 | 17 | — | 16 | 97 | — | — | — | — | — | — | — | |
| MAGEBaseline=GA+GD2026.04 | 16.5 | 31.65 | 54.95 | 50.3 | 48.94 | 0.96 | 0.43 | 50.96 | — | — | — | |
| GA + KLtraining=adapters only2025.10 | 13 | — | 20 | 97 | — | — | — | — | — | — | — | |
| TOFUBaseline=NPO2026.04 | 9.71 | 21.97 | 35.77 | 63.28 | 32.9 | 17.71 | 8.32 | 43.71 | — | — | — | |
| MAGEBaseline=GA+KL2026.04 | 5.41 | 51.74 | 57.27 | 47.41 | 54.67 | 2.27 | 1.21 | 54.49 | — | — | — | |
| SimNPO2026.06 | 5.27 | — | — | — | — | — | — | 61.4 | 97.6 | 99.2 | 46.4 | |
| GradDiff2026.06 | 3.3 | — | — | — | — | — | — | 56.1 | 55.6 | 73.9 | 46.4 | |
| MASC2026.06 | 3.01 | — | — | — | — | — | — | 66.6 | 89.9 | 83.2 | 44.6 | |
| NPO+KLR2026.06 | 2.99 | — | — | — | — | — | — | 51.6 | 72.2 | 34.2 | 43.4 | |
| MAGEBaseline=GA2026.04 | 2.86 | 27.99 | 54.57 | 46.35 | 45.31 | 0.59 | 0.28 | 49.48 | — | — | — | |
| MAGEBaseline=NPO2026.04 | 2.86 | 44.93 | 63.23 | 50.33 | 53.65 | 2.39 | 0.97 | 54.41 | — | — | — | |
| NPO2026.06 | 2.14 | — | — | — | — | — | — | 53.3 | 71.3 | 40.3 | 43 | |
| RMU2026.06 | 1.8 | — | — | — | — | — | — | 61.8 | 90.1 | 87.6 | 45.5 | |
| GA2026.06 | 1.72 | — | — | — | — | — | — | 45.9 | 73.2 | 18.6 | 45.5 | |
| DirectQABaseline=GA2026.04 | 1.43 | 56.84 | 43.37 | 55.3 | 50.33 | 24.18 | 19.48 | 55.86 | — | — | — | |
| Retrain2026.06 | 1 | — | — | — | — | — | — | 61.3 | 97.6 | 98.9 | 45.7 | |
| BeforeDescription=Pre-unlearning state2026.04 | 0.68 | 77.42 | 64.69 | 50.77 | 56.64 | 20.48 | 12.36 | 57.35 | — | — | — | |
| DirectQABaseline=NPO2026.04 | 0.68 | 77.31 | 64.52 | 50.94 | 56.49 | 20.88 | 12.71 | 57.33 | — | — | — | |
| DirectQABaseline=GA+GD2026.04 | 0.68 | 55.88 | 41.49 | 57.5 | 44.63 | 42.21 | 35.72 | 52.3 | — | — | — | |
| DirectQABaseline=GA+KL2026.04 | 0.68 | 69.29 | 58.73 | 51.38 | 55.76 | 20.68 | 13.57 | 57.49 | — | — | — | |
| Base/full2026.06 | 0 | — | — | — | — | — | — | 62.8 | 98.1 | 98.9 | 46 |