Machine Unlearning on WMDP-bio 1.0 (test)
80.3BF16 AccuracyZero-shot
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Zero-shotModel=Qwen-3-8B2026.05 | 80.3 | 80.3 | 0 | — | 74.1 | |
| Zero-shotModel=Qwen-3-4B2026.05 | 73.1 | 73.1 | 0 | — | 65.7 | |
| SimNPOModel=Qwen-3-4B2026.05 | 72.7 | 73.4 | 0.007 | 74.1 | 63.1 | |
| GU+SimNPOModel=Qwen-3-4B2026.05 | 72.4 | 72.7 | 0.003 | 73.8 | 63.1 | |
| NPOModel=Qwen-3-4B2026.05 | 71.8 | 70.7 | 0.011 | 73.1 | 62.4 | |
| NPOModel=Qwen-2.5-4B2026.05 | 70.7 | 69 | 0.018 | 71.3 | 58.4 | |
| SimNPOModel=Qwen-2.5-4B2026.05 | 70.3 | 69.7 | 0.007 | 71.5 | 59.1 | |
| GU+SimNPOModel=Qwen-2.5-4B2026.05 | 69.8 | 70 | 0.002 | 71.5 | 59.1 | |
| Zero-shotModel=Qwen-2.5-4B2026.05 | 69.3 | 69.3 | 0 | — | 61 | |
| Zero-shotModel=Llama-3.2-3B2026.05 | 68.7 | 68.7 | 0 | — | 53.8 | |
| SimNPOModel=Llama-3.2-3B2026.05 | 66.5 | 67.5 | 0.01 | 64 | 50.5 | |
| GU+SimNPOModel=Llama-3.2-3B2026.05 | 66.2 | 66.8 | 0.007 | 64.3 | 51.4 | |
| NPOModel=Llama-3.2-3B2026.05 | 64.8 | 63.2 | 0.017 | 65.2 | 50 | |
| MANSUModel=Qwen-3-8B2026.05 | 61.7 | 58.1 | 0.036 | 67.1 | 72.9 | |
| Surgical GAModel=Llama-3.1-8B2026.05 | 54.7 | 57.3 | 0.027 | 56 | 48.3 | |
| MANSUModel=Qwen-3-4B2026.05 | 52.7 | 49 | 0.037 | 61.7 | 64.4 | |
| MANSUModel=Qwen-2.5-4B2026.05 | 48.7 | 45.4 | 0.033 | 58.8 | 59.8 | |
| NPOModel=Llama-3.1-8B2026.05 | 44.3 | 42.3 | 0.02 | 50.3 | 56.3 | |
| MANSUModel=Llama-3.2-3B2026.05 | 43.4 | 40.4 | 0.03 | 52.8 | 52.1 | |
| MANSUModel=Llama-3.1-8B2026.05 | 43 | 39 | 0.04 | 52.3 | 57.3 | |
| NPOModel=Qwen-3-8B2026.05 | 28.3 | 32 | 0.037 | 30.3 | 49.2 | |
| GU+SimNPOModel=Qwen-3-8B2026.05 | 26.7 | 26.3 | 0.003 | 27.7 | 56.8 | |
| Global GAModel=Llama-3.1-8B2026.05 | 26 | 31 | 0.05 | 26 | 23.5 | |
| Surgical GAModel=Qwen-3-8B2026.05 | 26 | 24.7 | 0.013 | 30.3 | 45.8 | |
| SimNPOModel=Llama-3.1-8B2026.05 | 24.3 | 24.7 | 0.003 | 30.3 | 55.3 | |
| Global GAModel=Qwen-3-8B2026.05 | 23.3 | 24.3 | 0.007 | 24.7 | 24.2 | |
| GU+SimNPOModel=Llama-3.1-8B2026.05 | 23 | 23 | 0 | 24.7 | 20 | |
| SimNPOModel=Qwen-3-8B2026.05 | 22.7 | 22.7 | 0 | 25.7 | 26.5 |