LLM Unlearning on TOFU 5% (forget set)
-13.591FQFull
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| FullModel Size=3B, Category=Reference (Pre-unlearning)2026.01 | -13.591 | 0.666 | 88.7 | 88.4 | — | 0.85 | |
| Full (reference)Pretrained model=Phi-3.52026.01 | -13.232 | 0.693 | 86.8 | 83.5 | -0.033 | 0.866 | |
| GDModel Size=3B, Category=Training-based2026.01 | -13.232 | 0.552 | 14 | 24.4 | — | 0.579 | |
| NPOPretrained model=Phi-3.52026.01 | -12.877 | 0.278 | 53.8 | 59.4 | 0.057 | 0.855 | |
| FullModel Size=1B, Category=Reference (Pre-unlearning)2026.01 | -11.845 | 0.599 | 72.7 | 73.7 | — | 0.858 | |
| GAPretrained model=Phi-3.52026.01 | -11.511 | 0.073 | 2.7 | 2.8 | 0.001 | 0.822 | |
| GDPretrained model=Phi-3.52026.01 | -11.183 | 0.665 | 34.4 | 57.4 | 0.231 | 0.875 | |
| NPO+Pretrained model=Phi-3.52026.01 | -10.859 | 0.552 | 59.1 | 76.1 | 0.17 | 0.877 | |
| GDModel Size=1B, Category=Training-based2026.01 | -8.831 | 0.457 | 9 | 17.1 | — | 0.751 | |
| NPO+Model Size=3B, Category=Training-based2026.01 | -7.352 | 0.545 | 10 | 20 | — | 0.911 | |
| NPOModel Size=3B, Category=Training-based2026.01 | -7.091 | 0.472 | 8 | 14 | — | 0.868 | |
| GAModel Size=3B, Category=Training-based2026.01 | -5.856 | 0.482 | 8.9 | 13.5 | — | 0.866 | |
| TVModel Size=1B, Category=Task Arithmetic-based2026.01 | -5.623 | 0.478 | 4.9 | 14.8 | — | 0.94 | |
| TVModel Size=3B, Category=Task Arithmetic-based2026.01 | -5.395 | 0.628 | 5.3 | 21.4 | — | 0.926 | |
| NPO+Model Size=1B, Category=Training-based2026.01 | -4.26 | 0.458 | 9.8 | 13.9 | — | 0.882 | |
| PerTA-gradPretrained model=Phi-3.52026.01 | -3.548 | 0.667 | 10.7 | 41.2 | 0.305 | 0.879 | |
| GAModel Size=1B, Category=Training-based2026.01 | -2.415 | 0 | 3.7 | 3.9 | — | 0.417 | |
| NPOModel Size=1B, Category=Training-based2026.01 | -2.222 | 0 | 4.8 | 5.2 | — | 0.543 | |
| PerTA-gradModel Size=1B, Category=Task Arithmetic-based2026.01 | -0.661 | 0.546 | 6.9 | 31 | — | 0.91 | |
| PerTA-fisherModel Size=3B, Category=Task Arithmetic-based2026.01 | -0.405 | 0.677 | 8.3 | 56.1 | — | 0.906 | |
| PerTA-fisherModel Size=1B, Category=Task Arithmetic-based2026.01 | -0.339 | 0.553 | 7.7 | 34.8 | — | 0.911 | |
| PerTA-gradModel Size=3B, Category=Task Arithmetic-based2026.01 | -0.263 | 0.674 | 7.9 | 50.2 | — | 0.915 | |
| GT (reference)Pretrained model=Phi-3.52026.01 | 0 | 0.678 | 8.2 | 85.5 | 0.773 | 0.881 | |
| GTModel Size=1B, Category=Reference (Ground Truth)2026.01 | 0 | 0.599 | 6.3 | 74.6 | — | 0.905 | |
| GTModel Size=3B, Category=Reference (Ground Truth)2026.01 | 0 | 0.659 | 6.6 | 87.4 | — | 0.869 |