Unlearning on TOFU (Split99)
1Forget QualityUNDIAL
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| UNDIALModel=Qwen2.5-1.5B-Instruct, Setting=Clean2026.02 | 1 | 76.9 | 0.338 | 12.3 | |
| GRADASCENTModel=Qwen2.5-1.5B-Instruct, Setting=MPU2026.02 | 0.99 | 74 | 0.346 | -21.7 | |
| GRADDIFFModel=Qwen2.5-1.5B-Instruct, Setting=MPU2026.02 | 0.99 | 74 | 0.346 | -21.6 | |
| DPOModel=Qwen2.5-1.5B-Instruct, Setting=MPU2026.02 | 0.99 | 73.8 | 0.347 | -21.7 | |
| SIMNPOModel=Qwen2.5-1.5B-Instruct, Setting=MPU2026.02 | 0.99 | 73.8 | 0.347 | -22 | |
| SATIMPModel=Qwen2.5-1.5B-Instruct, Setting=MPU2026.02 | 0.99 | 74 | 0.346 | -21.5 | |
| NPOModel=Qwen2.5-1.5B-Instruct, Setting=MPU2026.02 | 0.919 | 73.9 | 0.346 | -21.9 | |
| NPOModel=Llama-3.2-1B, Setting=MPU, noise level (κ)=0.12026.02 | 0.919 | 61.8 | 0.596 | 28.1 | |
| DPOModel=Qwen2.5-3B-Instruct, Setting=MPU2026.02 | 0.766 | 70.9 | 0.397 | -28.8 | |
| NPOModel=Qwen2.5-3B-Instruct, Setting=MPU2026.02 | 0.766 | 71.1 | 0.397 | -28.5 | |
| SIMNPOModel=Qwen2.5-3B-Instruct, Setting=MPU2026.02 | 0.766 | 71 | 0.397 | -28.6 | |
| DPOModel=Llama-3.2-3B-Instruct, Setting=MPU2026.02 | 0.579 | 68.3 | 0.633 | -3.95 | |
| GRADASCENTModel=Qwen2.5-3B-Instruct, Setting=MPU2026.02 | 0.579 | 70.9 | 0.391 | -28.1 | |
| DPOModel=Qwen2.5-1.5B-Instruct, Setting=Clean2026.02 | 0.266 | 75.7 | 0.305 | 86 | |
| DPOModel=Llama-3.2-1B, Setting=MPU, noise level (κ)=0.052026.02 | 0.266 | 64.1 | 0.594 | -29.2 | |
| SIMNPOModel=Llama-3.2-3B-Instruct, Setting=MPU2026.02 | 0.0971 | 55.4 | 0.656 | -64.3 | |
| GRADASCENTModel=Qwen2.5-1.5B-Instruct, Setting=Noised2026.02 | 0.0541 | 53.6 | 0.0007 | 62.1 | |
| GRADDIFFModel=Llama-3.2-3B-Instruct, Setting=MPU2026.02 | 0.0541 | 48.5 | 0.409 | 107 | |
| GRADASCENTModel=Llama-3.2-1B, Setting=MPU, noise level (κ)=0.012026.02 | 0.0541 | 46.8 | 0.0002 | 69.6 | |
| SATIMPModel=Llama-3.2-3B-Instruct, Setting=MPU2026.02 | 0.0286 | 51.2 | 0.659 | -99.9 | |
| GRADASCENTModel=Qwen2.5-1.5B-Instruct, Setting=Clean2026.02 | 0.0143 | 51.6 | 0.001 | 13.7 | |
| NPOModel=Qwen2.5-1.5B-Instruct, Setting=Clean2026.02 | 0.0143 | 70.8 | 0.305 | 89.4 | |
| GRADASCENTModel=Llama-3.2-1B, Setting=MPU, noise level (κ)=02026.02 | 0.0068 | 44 | 0.0001 | 67.9 | |
| GRADDIFFModel=Qwen2.5-1.5B-Instruct, Setting=Clean2026.02 | 0.0001 | 62.2 | 0.297 | 80.9 | |
| GRADASCENTModel=Llama-3.2-3B-Instruct, Setting=Clean2026.02 | 0 | 18.7 | 0.0001 | 50.6 |