Machine Unlearning on TOFU 1.0 (Real Author)
93ROUGE-LLLAMA2-7B-TOFU
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| LLAMA2-7B-TOFUModel Type=Base (No Unlearning)2024.10 | 93 | 0.45 | 58 | |
| LLAMA2-13B-TOFUModel Type=Base (No Unlearning)2024.10 | 89 | 0.51 | 67 | |
| LoRABase Model=LLAMA2-7B-TOFU2024.10 | 88 | 0.51 | 68 | |
| LoRABase Model=LLAMA2-13B-TOFU2024.10 | 87 | 0.54 | 70 | |
| CMCBase Model=LLAMA2-7B-TOFU2024.10 | 84 | 0.43 | 58 | |
| MISTRAL-7B-TOFUModel Type=Base (No Unlearning)2024.10 | 84 | 0.61 | 75 | |
| LoRABase Model=MISTRAL-7B-TOFU2024.10 | 79 | 0.57 | 71 | |
| CMCBase Model=LLAMA2-13B-TOFU2024.10 | 77 | 0.49 | 65 | |
| CMCBase Model=MISTRAL-7B-TOFU2024.10 | 73 | 0.62 | 75 | |
| DPOBackbone=LLaDA-8B-Instruct2026.05 | 69.9 | 0.046 | — | |
| BaseBackbone=Dream-7B-Instruct2026.05 | 68 | 0.142 | — | |
| GDBackbone=LLaDA-8B-Instruct2026.05 | 64.5 | 0.053 | — | |
| MDUBackbone=LLaDA-8B-Instruct, τ=0.502026.05 | 64.5 | 0.133 | — | |
| SimNPOBackbone=LLaDA-8B-Instruct2026.05 | 64 | 0.055 | — | |
| MDUBackbone=LLaDA-8B-Instruct, τ=0.002026.05 | 62.9 | 0.093 | — | |
| MDUBackbone=LLaDA-8B-Instruct, τ=0.252026.05 | 61.6 | 0.116 | — | |
| MDUBackbone=LLaDA-8B-Instruct, τ=0.752026.05 | 61.2 | 0.155 | — | |
| BaseBackbone=LLaDA-8B-Instruct2026.05 | 61.1 | 0.041 | — | |
| δ-UNLEARNINGBase Model=LLAMA2-13B-TOFU2024.10 | 61 | 0.36 | 46 | |
| IHL + CETraining Mode=Adapters only, Data Balance=Remaining data equal to TUD2025.10 | 61 | — | 21 | |
| NPOBackbone=LLaDA-8B-Instruct2026.05 | 60.6 | 0.033 | — | |
| GABackbone=LLaDA-8B-Instruct2026.05 | 59.1 | 0.068 | — | |
| WGABackbone=LLaDA-8B-Instruct2026.05 | 57.7 | 0.089 | — | |
| MDUBackbone=LLaDA-8B-Instruct, τ=1.002026.05 | 56.8 | 0.11 | — | |
| MDUBackbone=Dream-7B-Instruct, τ=0.002026.05 | 54.7 | 0.055 | — | |
| WGABackbone=Dream-7B-Instruct2026.05 | 54.6 | 0.053 | — | |
| DPOBackbone=Dream-7B-Instruct2026.05 | 54.3 | 0.052 | — | |
| MDUBackbone=Dream-7B-Instruct, τ=0.252026.05 | 54.2 | 0.13 | — | |
| MDUBackbone=Dream-7B-Instruct, τ=0.502026.05 | 54.2 | 0.082 | — | |
| MDUBackbone=Dream-7B-Instruct, τ=1.002026.05 | 53.4 | 0.053 | — | |
| SimNPOBackbone=Dream-7B-Instruct2026.05 | 53.2 | 0.071 | — | |
| MDUBackbone=Dream-7B-Instruct, τ=0.752026.05 | 48.7 | 0.03 | — | |
| GDBackbone=Dream-7B-Instruct2026.05 | 48.1 | 0.103 | — | |
| NPOBackbone=Dream-7B-Instruct2026.05 | 44.1 | 0.048 | — | |
| NPO + CETraining Mode=Adapters only, Data Balance=Remaining data equal to TUD2025.10 | 35 | — | 89 | |
| IHL + KLTraining Mode=Adapters only, Data Balance=Remaining data equal to TUD2025.10 | 32 | — | 58 | |
| IHLTraining Mode=Adapters only, Data Balance=Remaining data equal to TUD2025.10 | 28 | — | 60 | |
| NPO + KLTraining Mode=Adapters only, Data Balance=Remaining data equal to TUD2025.10 | 26 | — | 93 | |
| GABackbone=Dream-7B-Instruct2026.05 | 24.9 | 0.018 | — | |
| NPOTraining Mode=Adapters only, Data Balance=Remaining data equal to TUD2025.10 | 20 | — | 92 | |
| ULDTraining Mode=Adapters only, Data Balance=Remaining data equal to TUD2025.10 | 18 | — | 53 | |
| GATraining Mode=Adapters only, Data Balance=Remaining data equal to TUD2025.10 | 12 | — | 95 | |
| GA + KLTraining Mode=Adapters only, Data Balance=Remaining data equal to TUD2025.10 | 10 | — | 97 | |
| GA + CETraining Mode=Adapters only, Data Balance=Remaining data equal to TUD2025.10 | 8 | — | 97 | |
| Attention-ShiftingTraining Mode=Adapters only, Data Balance=Remaining data equal to TUD2025.10 | 5 | — | 98 |