Machine Unlearning on WMDP-cyber 1.0 (test)
53.7BF16 ScoreZero-shot
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Zero-shotModel=Qwen-3-8B2026.05 | 53.7 | 53.7 | 0 | — | 74.1 | |
| Zero-shotModel=Qwen-3-4B2026.05 | 51.7 | 51.7 | 0 | — | 65.7 | |
| MANSUModel=Qwen-3-8B2026.05 | 49.7 | 46.4 | 0.033 | 54.1 | 72.1 | |
| GU+SimNPOModel=Qwen-3-4B2026.05 | 49.1 | 49.4 | 0.003 | 48.7 | 63.1 | |
| Zero-shotModel=Qwen-2.5-4B2026.05 | 48.7 | 48.7 | 0 | — | 61 | |
| SimNPOModel=Qwen-3-4B2026.05 | 48.7 | 49.7 | 0.01 | 48.4 | 63.1 | |
| NPOModel=Qwen-3-4B2026.05 | 48.1 | 47.7 | 0.004 | 47.7 | 61.8 | |
| GU+SimNPOModel=Qwen-2.5-4B2026.05 | 45.5 | 46.3 | 0.008 | 45.3 | 58.8 | |
| NPOModel=Qwen-2.5-4B2026.05 | 44.7 | 44.5 | 0.002 | 44.5 | 58 | |
| SimNPOModel=Qwen-2.5-4B2026.05 | 44 | 46.8 | 0.028 | 44.8 | 58 | |
| Surgical GAModel=Qwen-3-8B2026.05 | 43 | 46.7 | 0.037 | 47.3 | 71 | |
| GU+SimNPOModel=Llama-3.2-3B2026.05 | 42 | 42.3 | 0.003 | 43 | 51.4 | |
| SimNPOModel=Llama-3.2-3B2026.05 | 41.8 | 42.2 | 0.003 | 43.3 | 51.8 | |
| Zero-shotModel=Llama-3.2-3B2026.05 | 40.3 | 40.3 | 0 | — | 53.8 | |
| NPOModel=Qwen-3-8B2026.05 | 36 | 39.3 | 0.033 | 43.7 | 71.5 | |
| Global GAModel=Llama-3.1-8B2026.05 | 35.7 | 37 | 0.013 | 39 | 54.3 | |
| NPOModel=Llama-3.1-8B2026.05 | 34 | 34.3 | 0.003 | 40 | 56.8 | |
| GU+SimNPOModel=Qwen-3-8B2026.05 | 33.3 | 44.3 | 0.11 | 26.7 | 71.5 | |
| NPOModel=Llama-3.2-3B2026.05 | 32.3 | 33.8 | 0.015 | 34.3 | 47.3 | |
| MANSUModel=Llama-3.1-8B2026.05 | 32.3 | 31.3 | 0.01 | 39.1 | 58.6 | |
| SimNPOModel=Llama-3.1-8B2026.05 | 32 | 30.7 | 0.013 | 39 | 51 | |
| MANSUModel=Qwen-3-4B2026.05 | 31.4 | 28.7 | 0.027 | 41.4 | 64.4 | |
| GU+SimNPOModel=Llama-3.1-8B2026.05 | 30 | 30 | 0 | 23 | 29.7 | |
| MANSUModel=Qwen-2.5-4B2026.05 | 29.1 | 26.8 | 0.023 | 38.4 | 59.8 | |
| Surgical GAModel=Llama-3.1-8B2026.05 | 28.3 | 29 | 0.007 | 36.7 | 48 | |
| SimNPOModel=Qwen-3-8B2026.05 | 27 | 40.3 | 0.133 | 28 | 55.5 | |
| MANSUModel=Llama-3.2-3B2026.05 | 25.4 | 23.4 | 0.02 | 34.8 | 52.1 | |
| Global GAModel=Qwen-3-8B2026.05 | 21.3 | 44.7 | 0.233 | 24.7 | 71 |