Machine Unlearning on TOFU (Split99)
1Forget QualityUNDIAL
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| UNDIALBackbone=Qwen2.5-1.5B-Instruct, Setting=CLEAN, m=2, kappa=0.012026.02 | 1 | 76.9 | 0.338 | 12.3 | |
| GRADASCENTBackbone=Qwen2.5-1.5B-Instruct, Setting=MPU, m=2, kappa=0.012026.02 | 0.99 | 74 | 0.346 | -21.7 | |
| GRADDIFFBackbone=Qwen2.5-1.5B-Instruct, Setting=MPU, m=2, kappa=0.012026.02 | 0.99 | 74 | 0.346 | -21.6 | |
| DPOBackbone=Qwen2.5-1.5B-Instruct, Setting=MPU, m=2, kappa=0.012026.02 | 0.99 | 73.8 | 0.347 | -21.7 | |
| SIMNPOBackbone=Qwen2.5-1.5B-Instruct, Setting=CLEAN, m=2, kappa=0.012026.02 | 0.99 | 73.8 | 0.349 | -18.7 | |
| SIMNPOBackbone=Qwen2.5-1.5B-Instruct, Setting=MPU, m=2, kappa=0.012026.02 | 0.99 | 73.8 | 0.347 | -22 | |
| UNDIALBackbone=Qwen2.5-1.5B-Instruct, Setting=MPU, m=2, kappa=0.012026.02 | 0.99 | 74 | 0.346 | -21.8 | |
| SATIMPBackbone=Qwen2.5-1.5B-Instruct, Setting=MPU, m=2, kappa=0.012026.02 | 0.99 | 74 | 0.346 | -21.5 | |
| NPOBackbone=Qwen2.5-1.5B-Instruct, Setting=MPU, m=2, kappa=0.012026.02 | 0.919 | 73.9 | 0.346 | -21.9 | |
| SATIMPBackbone=Qwen2.5-1.5B-Instruct, Setting=CLEAN, m=2, kappa=0.012026.02 | 0.919 | 73.8 | 0.349 | -21.8 | |
| GRADDIFFBackbone=Qwen2.5-3B-Instruct, Protocol=CLEAN, m=2, κ=0.012026.02 | 0.766 | 71.2 | 0.394 | -28 | |
| GRADDIFFBackbone=Qwen2.5-3B-Instruct, Protocol=MPU, m=2, κ=0.012026.02 | 0.766 | 71 | 0.395 | -27.1 | |
| DPOBackbone=Qwen2.5-3B-Instruct, Protocol=CLEAN, m=2, κ=0.012026.02 | 0.766 | 71.1 | 0.397 | -28.9 | |
| DPOBackbone=Qwen2.5-3B-Instruct, Protocol=MPU, m=2, κ=0.012026.02 | 0.766 | 70.9 | 0.397 | -28.8 | |
| NPOBackbone=Qwen2.5-3B-Instruct, Protocol=CLEAN, m=2, κ=0.012026.02 | 0.766 | 71.2 | 0.398 | -28.5 | |
| NPOBackbone=Qwen2.5-3B-Instruct, Protocol=MPU, m=2, κ=0.012026.02 | 0.766 | 71.1 | 0.397 | -28.5 | |
| SIMNPOBackbone=Qwen2.5-3B-Instruct, Protocol=CLEAN, m=2, κ=0.012026.02 | 0.766 | 71.2 | 0.398 | -28.5 | |
| SIMNPOBackbone=Qwen2.5-3B-Instruct, Protocol=MPU, m=2, κ=0.012026.02 | 0.766 | 71 | 0.397 | -28.6 | |
| UNDIALBackbone=Qwen2.5-3B-Instruct, Protocol=CLEAN, m=2, κ=0.012026.02 | 0.766 | 71 | 0.398 | -28.1 | |
| UNDIALBackbone=Qwen2.5-3B-Instruct, Protocol=MPU, m=2, κ=0.012026.02 | 0.766 | 71.1 | 0.396 | -28.8 | |
| SATIMPBackbone=Qwen2.5-3B-Instruct, Protocol=CLEAN, m=2, κ=0.012026.02 | 0.766 | 71.1 | 0.397 | -28.9 | |
| SATIMPBackbone=Qwen2.5-3B-Instruct, Protocol=MPU, m=2, κ=0.012026.02 | 0.766 | 71.2 | 0.397 | -28.7 | |
| GRADASCENTBackbone=Qwen2.5-3B-Instruct, Protocol=CLEAN, m=2, κ=0.012026.02 | 0.579 | 70.9 | 0.392 | -28.7 | |
| GRADASCENTBackbone=Qwen2.5-3B-Instruct, Protocol=MPU, m=2, κ=0.012026.02 | 0.579 | 70.9 | 0.391 | -28.1 | |
| DPOBackbone=Qwen2.5-1.5B-Instruct, Setting=CLEAN, m=2, kappa=0.012026.02 | 0.266 | 75.7 | 0.305 | 86 | |
| GRADASCENTBackbone=Qwen2.5-1.5B-Instruct, Setting=CLEAN, m=2, kappa=0.012026.02 | 0.0143 | 51.6 | 0.001 | 13.7 | |
| NPOBackbone=Qwen2.5-1.5B-Instruct, Setting=CLEAN, m=2, kappa=0.012026.02 | 0.0143 | 70.8 | 0.305 | 89.4 | |
| GRADDIFFBackbone=Qwen2.5-1.5B-Instruct, Setting=CLEAN, m=2, kappa=0.012026.02 | 0.0001 | 62.2 | 0.297 | 80.9 |