Machine Unlearning on TOFU 1.0 (forget01)
80.1Average ScoreASU
Evaluation results
| Method | Links | |||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ASUVariant=KL2026.03 | 80.1 | 77.13 | 83.08 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ASUVariant=GD2026.03 | 79.5 | 76.79 | 82.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ASU_KLTarget Model=Llama-3.1-8B2026.03 | 78.02 | 78.36 | 77.69 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ME_KLTarget Model=Llama-3.1-8B, Method category=Convergence-based2026.03 | 75.99 | 78.88 | 73.09 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MEVariant=GD2026.03 | 73.76 | 72.48 | 75.04 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MEVariant=KL2026.03 | 70.43 | 73.82 | 67.04 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GAVariant=KL2026.03 | 68.28 | 67.83 | 68.73 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GAVariant=GD2026.03 | 68.02 | 66.59 | 69.46 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| IDKVariant=AP2026.03 | 67.99 | 75.69 | 60.29 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| NPOVariant=GD2026.03 | 67.62 | 64.1 | 71.14 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| NPOVariant=KL2026.03 | 67.45 | 64.19 | 70.71 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GUARD-ITModel=Llama-3.2-3B-Instruct2026.05 | 66 | — | — | — | — | — | — | — | — | — | — | — | 67 | 66 | 84 | — | — | — | — | |
| IDKVariant=KL2026.03 | 64.15 | 67.14 | 61.16 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GUARD-ITModel=Llama-3.2-1B-Instruct2026.05 | 64 | — | — | — | — | — | — | — | — | — | — | — | 60 | 68 | 80 | — | — | — | — | |
| IDKVariant=GD2026.03 | 63.99 | 66.94 | 61.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| NPO_KLTarget Model=Llama-3.1-8B, Method category=Divergence-based2026.03 | 63.55 | 68.2 | 58.89 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| IDK_KLTarget Model=Llama-3.1-8B, Method category=Convergence-based2026.03 | 63.31 | 73.67 | 52.95 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| IDK_APTarget Model=Llama-3.1-8B, Method category=Convergence-based2026.03 | 62.48 | 77.68 | 47.28 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DPO_KLTarget Model=Llama-3.1-8B, Method category=Convergence-based2026.03 | 61.33 | 78.45 | 44.22 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DPOVariant=GD2026.03 | 59.29 | 75.68 | 42.91 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DPOVariant=KL2026.03 | 59.16 | 75.63 | 42.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GradDiffModel=Llama-3.2-3B-Instruct2026.05 | 57 | — | — | — | — | — | — | — | — | — | — | — | 44 | 78 | 30 | — | — | — | — | |
| GA_KLTarget Model=Llama-3.1-8B, Method category=Divergence-based2026.03 | 55.62 | 36.25 | 74.98 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| UNDIALModel=Llama-3.2-3B-Instruct2026.05 | 53 | — | — | — | — | — | — | — | — | — | — | — | 65 | 45 | 84 | — | — | — | — | |
| WGAModel=Llama-3.2-3B-Instruct2026.05 | 51 | — | — | — | — | — | — | — | — | — | — | — | 63 | 43 | 65 | — | — | — | — | |
| UNDIALModel=Llama-3.2-1B-Instruct2026.05 | 50 | — | — | — | — | — | — | — | — | — | — | — | 54 | 47 | 90 | — | — | — | — | |
| CEUModel=Llama-3.2-3B-Instruct2026.05 | 50 | — | — | — | — | — | — | — | — | — | — | — | 57 | 44 | 43 | — | — | — | — | |
| WGAModel=Llama-3.2-1B-Instruct2026.05 | 49 | — | — | — | — | — | — | — | — | — | — | — | 57 | 43 | 75 | — | — | — | — | |
| SimNPOModel=Llama-3.2-3B-Instruct2026.05 | 48 | — | — | — | — | — | — | — | — | — | — | — | 52 | 44 | 68 | — | — | — | — | |
| SatImpModel=Llama-3.2-1B-Instruct2026.05 | 46 | — | — | — | — | — | — | — | — | — | — | — | 60 | 38 | 86 | — | — | — | — | |
| NPOModel=Llama-3.2-1B-Instruct2026.05 | 45 | — | — | — | — | — | — | — | — | — | — | — | 51 | 40 | 80 | — | — | — | — | |
| NPO (KLR)Model=Llama-3.2-1B-Instruct2026.05 | 45 | — | — | — | — | — | — | — | — | — | — | — | 51 | 40 | 84 | — | — | — | — | |
| NPOModel=Llama-3.2-3B-Instruct2026.05 | 45 | — | — | — | — | — | — | — | — | — | — | — | 59 | 37 | 87 | — | — | — | — | |
| NPO (KLR)Model=Llama-3.2-3B-Instruct2026.05 | 45 | — | — | — | — | — | — | — | — | — | — | — | 58 | 37 | 90 | — | — | — | — | |
| GAModel=Llama-3.2-1B-Instruct2026.05 | 43 | — | — | — | — | — | — | — | — | — | — | — | 41 | 44 | 61 | — | — | — | — | |
| SimNPOModel=Llama-3.2-1B-Instruct2026.05 | 43 | — | — | — | — | — | — | — | — | — | — | — | 40 | 47 | 63 | — | — | — | — | |
| CEUModel=Llama-3.2-1B-Instruct2026.05 | 42 | — | — | — | — | — | — | — | — | — | — | — | 41 | 44 | 88 | — | — | — | — | |
| SatImpModel=Llama-3.2-3B-Instruct2026.05 | 42 | — | — | — | — | — | — | — | — | — | — | — | 65 | 31 | 84 | — | — | — | — | |
| GradDiffModel=Llama-3.2-1B-Instruct2026.05 | 41 | — | — | — | — | — | — | — | — | — | — | — | 38 | 43 | 63 | — | — | — | — | |
| GAModel=Llama-3.2-3B-Instruct2026.05 | 41 | — | — | — | — | — | — | — | — | — | — | — | 64 | 30 | 87 | — | — | — | — | |
| GradDiff (KLR)Model=Llama-3.2-1B-Instruct2026.05 | 40 | — | — | — | — | — | — | — | — | — | — | — | 58 | 30 | 88 | — | — | — | — | |
| SimNPO (KLR)Model=Llama-3.2-1B-Instruct2026.05 | 40 | — | — | — | — | — | — | — | — | — | — | — | 36 | 45 | 59 | — | — | — | — | |
| SimNPO (KLR)Model=Llama-3.2-3B-Instruct2026.05 | 40 | — | — | — | — | — | — | — | — | — | — | — | 35 | 48 | 30 | — | — | — | — | |
| Base2026.03 | 39.45 | 75.81 | 3.09 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GradDiff (KLR)Model=Llama-3.2-3B-Instruct2026.05 | 37 | — | — | — | — | — | — | — | — | — | — | — | 66 | 26 | 89 | — | — | — | — | |
| DPOModel=Llama-3.2-1B-Instruct2026.05 | 34 | — | — | — | — | — | — | — | — | — | — | — | 53 | 25 | 92 | — | — | — | — | |
| DPOModel=Llama-3.2-3B-Instruct2026.05 | 34 | — | — | — | — | — | — | — | — | — | — | — | 40 | 30 | 97 | — | — | — | — | |
| RMUModel=Llama-3.2-1B-Instruct2026.05 | 20 | — | — | — | — | — | — | — | — | — | — | — | 55 | 12 | 91 | — | — | — | — | |
| RMU (KLR)Model=Llama-3.2-1B-Instruct2026.05 | 20 | — | — | — | — | — | — | — | — | — | — | — | 58 | 12 | 88 | — | — | — | — | |
| RMUModel=Llama-3.2-3B-Instruct2026.05 | 14 | — | — | — | — | — | — | — | — | — | — | — | 63 | 8 | 88 | — | — | — | — | |
| RMU (KLR)Model=Llama-3.2-3B-Instruct2026.05 | 14 | — | — | — | — | — | — | — | — | — | — | — | 67 | 8 | 89 | — | — | — | — | |
| PDUModel=Llama-3.2-1B-Instruct2026.05 | 12 | — | — | — | — | — | — | — | — | — | — | — | 7 | 44 | 6 | — | — | — | — | |
| PDUModel=Llama-3.2-3B-Instruct2026.05 | 2 | — | — | — | — | — | — | — | — | — | — | — | 1 | 29 | 3 | — | — | — | — | |
| CBDAccess Level=Black-box2026.06 | — | 75.31 | — | — | — | — | — | — | — | — | — | — | — | — | — | 34.36 | 26.22 | 79.84 | 2.49 | |
| DPOTuning Strategy=Full Fine-tuning Methods, Backbone=Phi-1.5B2025.09 | — | 52 | — | 84 | 51 | 0.0472 | 87 | 47 | 43 | 45 | 76 | 52 | — | — | — | — | — | — | — | |
| DPOAccess Level=White-box2026.06 | — | 67.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | 29.46 | 63.14 | 59.33 | 1.45 | |
| DPO+GDAccess Level=White-box2026.06 | — | 67.26 | — | — | — | — | — | — | — | — | — | — | — | — | — | 25.83 | 62.49 | 60.1 | 1.52 | |
| DPO+KLAccess Level=White-box2026.06 | — | 67.31 | — | — | — | — | — | — | — | — | — | — | — | — | — | 27.78 | 63.55 | 59.26 | 1.47 | |
| GATuning Strategy=Full Fine-tuning Methods, Backbone=Phi-1.5B2025.09 | — | 51 | — | 67 | 49 | 0.0056 | 87 | 48 | 42 | 45 | 75 | 49 | — | — | — | — | — | — | — | |
| GAAccess Level=White-box2026.06 | — | 64.45 | — | — | — | — | — | — | — | — | — | — | — | — | — | 42.86 | 39.32 | 54.01 | 1.57 | |
| GA+FILATuning Strategy=Parameter-Efficient Methods, Backbone=Phi-1.5B, LoRA Rank=322025.09 | — | 0 | — | 3 | 78 | 0.0006 | 2 | 16 | 0 | 27 | 1 | 28 | — | — | — | — | — | — | — | |
| GA+GDAccess Level=White-box2026.06 | — | 57.35 | — | — | — | — | — | — | — | — | — | — | — | — | — | 26.01 | 8.1 | 53.78 | 3.36 | |
| GA+KLAccess Level=White-box2026.06 | — | 63.93 | — | — | — | — | — | — | — | — | — | — | — | — | — | 37.17 | 26.86 | 53.4 | 2 | |
| GDTuning Strategy=Full Fine-tuning Methods, Backbone=Phi-1.5B2025.09 | — | 52 | — | 68 | 48 | 0.0089 | 90 | 49 | 40 | 45 | 75 | 50 | — | — | — | — | — | — | — | |
| GD+FILATuning Strategy=Parameter-Efficient Methods, Backbone=Phi-1.5B, LoRA Rank=322025.09 | — | 0 | — | 4 | 77 | 0.115 | 3 | 17 | 0 | 24 | 2 | 26 | — | — | — | — | — | — | — | |
| GD+SineTuning Strategy=Parameter-Efficient Methods, Backbone=Phi-1.5B, LoRA Rank=322025.09 | — | 52 | — | 35 | 47 | 94.3 | 93 | 48 | 41 | 46 | 77 | 49 | — | — | — | — | — | — | — | |
| IHLTuning Strategy=Full Fine-tuning Methods, Backbone=Phi-1.5B2025.09 | — | 52 | — | 65 | 48 | 0.0128 | 90 | 49 | 42 | 45 | 76 | 50 | — | — | — | — | — | — | — | |
| KLTuning Strategy=Full Fine-tuning Methods, Backbone=Phi-1.5B2025.09 | — | 52 | — | 68 | 48 | 0.0042 | 87 | 49 | 43 | 45 | 77 | 49 | — | — | — | — | — | — | — | |
| LoKUTuning Strategy=Parameter-Efficient Methods, Backbone=Phi-1.5B, LoRA Rank=322025.09 | — | 52 | — | 37 | 61 | 3.06 | 71 | 49 | 43 | 47 | 73 | 53 | — | — | — | — | — | — | — | |
| NPOTuning Strategy=Full Fine-tuning Methods, Backbone=Phi-1.5B2025.09 | — | 51 | — | 65 | 49 | 0.006 | 87 | 48 | 42 | 44 | 75 | 49 | — | — | — | — | — | — | — | |
| NPOAccess Level=White-box2026.06 | — | 55.95 | — | — | — | — | — | — | — | — | — | — | — | — | — | 34.25 | 34.46 | 59.91 | 1.63 | |
| NPO+GDAccess Level=White-box2026.06 | — | 60.92 | — | — | — | — | — | — | — | — | — | — | — | — | — | 33.32 | 38.12 | 59.68 | 1.71 | |
| NPO+KLAccess Level=White-box2026.06 | — | 65.17 | — | — | — | — | — | — | — | — | — | — | — | — | — | 37.66 | 44.67 | 60.28 | 1.58 | |
| OffsetAccess Level=Gray-box2026.06 | — | 37.89 | — | — | — | — | — | — | — | — | — | — | — | — | — | 20.28 | 39.49 | 11.45 | 1.27 | |
| OriginalBackbone=Phi-1.5B2025.09 | — | 52 | — | 93 | 48 | 0 | 92 | 48 | 41 | 45 | 75 | 50 | — | — | — | — | — | — | — | |
| Retain90Backbone=Phi-1.5B2025.09 | — | 52 | — | 43 | 63 | 100 | 91 | 48 | 43 | 45 | 76 | 49 | — | — | — | — | — | — | — | |
| Retrain LLMAccess Level=Standard2026.06 | — | 73.86 | — | — | — | — | — | — | — | — | — | — | — | — | — | 40.76 | 33.79 | 69.42 | 1.98 | |
| Target LLMAccess Level=Standard2026.06 | — | 75.37 | — | — | — | — | — | — | — | — | — | — | — | — | — | 89.16 | 76.37 | 54.54 | 0.91 | |
| ULDAccess Level=Gray-box2026.06 | — | 76.08 | — | — | — | — | — | — | — | — | — | — | — | — | — | 12.75 | 21.93 | 64.56 | 4.39 |