Machine Unlearning on TOFU 1.0 (Forget05)
75.37Model Utility (MU)Target LLM
Evaluation results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Target LLMAccess Level=Standard2026.06 | 75.37 | — | — | — | — | — | — | — | — | — | 89.53 | 79.2 | 51.97 | 0.89 | |
| CBDAccess Level=Black-box2026.06 | 74.76 | — | — | — | — | — | — | — | — | — | 32.95 | 25.59 | 78.66 | 2.55 | |
| Retrain LLMAccess Level=Standard2026.06 | 74 | — | — | — | — | — | — | — | — | — | 39.49 | 34.39 | 69.3 | 2 | |
| DPO+KLAccess Level=White-box2026.06 | 66.34 | — | — | — | — | — | — | — | — | — | 37.15 | 66.76 | 55.86 | 1.28 | |
| DPO+GDAccess Level=White-box2026.06 | 65.66 | — | — | — | — | — | — | — | — | — | 30.07 | 67.02 | 55.84 | 1.35 | |
| ULDAccess Level=Gray-box2026.06 | 64.44 | — | — | — | — | — | — | — | — | — | 17.88 | 72.93 | 56.45 | 1.42 | |
| NPO+GDAccess Level=White-box2026.06 | 61 | — | — | — | — | — | — | — | — | — | 30.68 | 50.14 | 61.61 | 1.51 | |
| DPOAccess Level=White-box2026.06 | 60.57 | — | — | — | — | — | — | — | — | — | 19.43 | 65.11 | 56.43 | 1.43 | |
| NPO+KLAccess Level=White-box2026.06 | 56.44 | — | — | — | — | — | — | — | — | — | 26.38 | 45.4 | 63.75 | 1.57 | |
| GA+KLAccess Level=White-box2026.06 | 52.79 | — | — | — | — | — | — | — | — | — | 41.14 | 61.96 | 49.71 | 1.02 | |
| GA+GDAccess Level=White-box2026.06 | 51.23 | — | — | — | — | — | — | — | — | — | 37 | 49.62 | 51.54 | 1.18 | |
| NPOAccess Level=White-box2026.06 | 49.28 | — | — | — | — | — | — | — | — | — | 35.55 | 65.53 | 53.44 | 0.98 | |
| GAAccess Level=White-box2026.06 | 44.96 | — | — | — | — | — | — | — | — | — | 31.96 | 52.84 | 53.76 | 1.06 | |
| OffsetAccess Level=Gray-box2026.06 | 35.06 | — | — | — | — | — | — | — | — | — | 25.91 | 69.34 | 25.38 | 0.74 | |
| GD+SineTuning Strategy=Parameter-Efficient Methods, Backbone=Phi-1.5B, LoRA Rank=322025.09 | 0.52 | 0.33 | 0.47 | 0.284 | 0.93 | 0.48 | 0.43 | 0.46 | 0.75 | 0.49 | — | — | — | — | |
| IHLTuning Strategy=Full Fine-tuning Methods, Backbone=Phi-1.5B2025.09 | 0.51 | 0.45 | 0.5 | 0 | 0.79 | 0.49 | 0.38 | 0.46 | 0.71 | 0.5 | — | — | — | — | |
| LoKUTuning Strategy=Parameter-Efficient Methods, Backbone=Phi-1.5B, LoRA Rank=322025.09 | 0.5 | 0.34 | 0.6 | 0.003 | 0.71 | 0.48 | 0.37 | 0.46 | 0.69 | 0.52 | — | — | — | — | |
| DPOTuning Strategy=Full Fine-tuning Methods, Backbone=Phi-1.5B2025.09 | 0.49 | 0.35 | 0.49 | 0 | 0.76 | 0.47 | 0.34 | 0.43 | 0.72 | 0.5 | — | — | — | — | |
| NPOTuning Strategy=Full Fine-tuning Methods, Backbone=Phi-1.5B2025.09 | 0.36 | 0.45 | 0.61 | 0 | 0.46 | 0.38 | 0.37 | 0.4 | 0.68 | 0.43 | — | — | — | — | |
| GDTuning Strategy=Full Fine-tuning Methods, Backbone=Phi-1.5B2025.09 | 0.23 | 0.24 | 0.56 | 0.0018 | 0.32 | 0.44 | 0.06 | 0.41 | 0.39 | 0.43 | — | — | — | — | |
| KLTuning Strategy=Full Fine-tuning Methods, Backbone=Phi-1.5B2025.09 | 0 | 0 | 0.76 | 0 | 0.01 | 0.16 | 0 | 0.26 | 0 | 0.26 | — | — | — | — | |
| GATuning Strategy=Full Fine-tuning Methods, Backbone=Phi-1.5B2025.09 | 0 | 0 | 0.76 | 0 | 0.01 | 0.16 | 0 | 0.26 | 0 | 0.25 | — | — | — | — | |
| GA+FILATuning Strategy=Parameter-Efficient Methods, Backbone=Phi-1.5B, LoRA Rank=322025.09 | 0 | 0 | 0.22 | 0 | 0 | 0.35 | 0 | 0.35 | 0 | 0.37 | — | — | — | — | |
| GD+FILATuning Strategy=Parameter-Efficient Methods, Backbone=Phi-1.5B, LoRA Rank=322025.09 | 0 | 0.04 | 0.71 | 0 | 0.05 | 0.17 | 0 | 0.23 | 0.02 | 0.28 | — | — | — | — |