Machine Unlearning on WMDP Cyber (test)
61.15MMLUOriginal
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| OriginalBackbone Model=Llama-3.1-8B2025.08 | 61.15 | — | — | — | — | — | 61.32 | 40.95 | 54 | 1.27 | 1.43 | |
| RMUBackbone Model=Llama-3.1-8B2025.08 | 61.15 | — | — | — | — | — | 52.47 | 33.7 | 55 | 0.68 | 1.23 | |
| DUETForgetting Data (Df)=D^query_f, Retention Data (Dr)=Dr2026.01 | 60.65 | — | — | — | — | 26.6 | — | — | — | — | — | |
| Refusal TrainingForgetting Data (Df)=D^QR_f, Retention Data (Dr)=Dr2026.01 | 60.63 | — | — | — | — | 40.92 | — | — | — | — | — | |
| GARetention Data (Dr)=Yes2026.01 | 59.82 | — | — | — | — | 40.36 | — | — | — | — | — | |
| CRISPBackbone Model=Llama-3.1-8B2025.08 | 58.86 | — | — | — | — | — | 61.74 | 29.38 | 53 | 1.14 | 1.49 | |
| ELMBackbone Model=Llama-3.1-8B2025.08 | 58.56 | — | — | — | — | — | 58.91 | 30.78 | 53 | 0.99 | 1.4 | |
| Base ModelBackbone=Zephyr-7B2026.01 | 58.12 | — | — | — | — | 43.68 | — | — | — | — | — | |
| RMURetention Data (Dr)=Yes2026.01 | 57.81 | — | — | — | — | 26.93 | — | — | — | — | — | |
| NPORetention Data (Dr)=Yes2026.01 | 57.11 | — | — | — | — | 39.61 | — | — | — | — | — | |
| NPO2026.01 | 55.34 | — | — | — | — | 36.89 | — | — | — | — | — | |
| SimNPO2026.01 | 54.25 | — | — | — | — | 34.22 | — | — | — | — | — | |
| GA2026.01 | 48.79 | — | — | — | — | 33.77 | — | — | — | — | — | |
| OriginalBackbone Model=Gemma-2-2B2025.08 | 46.3 | — | — | — | — | — | 52.57 | 33.9 | 39 | 1.05 | 1.46 | |
| CRISPBackbone Model=Gemma-2-2B2025.08 | 46.26 | — | — | — | — | — | 49.02 | 27.26 | 38 | 0.81 | 1.28 | |
| RMUBackbone Model=Gemma-2-2B2025.08 | 44.79 | — | — | — | — | — | 44.79 | 28.67 | 36 | 0.64 | 1.23 | |
| ELMBackbone Model=Gemma-2-2B2025.08 | 38.71 | — | — | — | — | — | 43.33 | 28.87 | 29 | 0.76 | 1.36 | |
| RMU2026.01 | 25.5 | — | — | — | — | 24.61 | — | — | — | — | — | |
| FLAT2026.01 | 23.24 | — | — | — | — | 24.51 | — | — | — | — | — | |
| GA + KLRetention data=D_retain2026.02 | 0.5982 | 0.4036 | — | — | 5.36 | — | — | — | — | — | — | |
| Base modelBackbone=Zephyr 7B beta [51]2026.02 | 0.581 | 0.44 | — | — | — | — | — | — | — | — | — | |
| RMURetention data=D_retain2026.02 | 0.5781 | 0.2693 | — | — | 16.78 | — | — | — | — | — | — | |
| NPO + KLRetention data=D_retain2026.02 | 0.5711 | 0.3961 | — | — | 3.4 | — | — | — | — | — | — | |
| NPORetention data=None2026.02 | 0.5534 | 0.3689 | — | — | 4.35 | — | — | — | — | — | — | |
| SimNPORetention data=None2026.02 | 0.5425 | 0.3422 | — | — | 5.93 | — | — | — | — | — | — | |
| CATNIPRetention data=None2026.02 | 0.5301 | 0.2869 | — | — | 10.22 | — | — | — | — | — | — | |
| GARetention data=None2026.02 | 0.4879 | 0.3377 | — | — | 0.92 | — | — | — | — | — | — | |
| RMU*Retention data=None2026.02 | 0.255 | 0.2461 | — | — | -13.21 | — | — | — | — | — | — | |
| FLATContrastive data=D_ct2026.02 | 0.2324 | 0.2451 | — | — | -15.37 | — | — | — | — | — | — |