Unlearning on WMDP (forget split)
55.4BF16 PrecisionGU+SimNPO
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| GU+SimNPOFamily=Qwen, n=92026.05 | 55.4 | 56.6 | 0.012 | |
| SimNPOFamily=Qwen, n=92026.05 | 54.1 | 56.6 | 0.025 | |
| NPOFamily=Qwen, n=92026.05 | 53.7 | 53.1 | 0.005 | |
| SimNPOFamily=Llama, n=62026.05 | 48.7 | 47.5 | 0.009 | |
| NPOFamily=Overall, n=242026.05 | 48.5 | 47.6 | 0.009 | |
| GU+SimNPOFamily=Overall, n=242026.05 | 46.7 | 47.2 | 0.006 | |
| SimNPOFamily=Overall, n=242026.05 | 46.5 | 46.9 | 0.005 | |
| NPOFamily=Llama, n=62026.05 | 44.9 | 43.8 | 0.013 | |
| MANSUFamily=Qwen, n=92026.05 | 43.2 | 40 | 0.032 | |
| Surgical GAFamily=Llama, n=32026.05 | 41.9 | 42.9 | 0.01 | |
| GU+SimNPOFamily=Llama, n=62026.05 | 39 | 39.1 | 0.001 | |
| Surgical GAFamily=Overall, n=62026.05 | 37.7 | 38.6 | 0.01 | |
| NPOFamily=Gemma, n=92026.05 | 37.1 | 36 | 0.011 | |
| Global GAFamily=Llama, n=32026.05 | 37 | 38.4 | 0.014 | |
| SimNPOFamily=Gemma, n=92026.05 | 36.6 | 36.6 | 0 | |
| MANSUFamily=Llama, n=62026.05 | 35.9 | 33.3 | 0.025 | |
| MANSUFamily=Overall, n=242026.05 | 35.9 | 33.2 | 0.026 | |
| GU+SimNPOFamily=Gemma, n=92026.05 | 35.5 | 36 | 0.005 | |
| Surgical GAFamily=Qwen, n=32026.05 | 33.4 | 34.4 | 0.009 | |
| Global GAFamily=Overall, n=62026.05 | 29.9 | 35.6 | 0.057 | |
| MANSUFamily=Gemma, n=92026.05 | 28.5 | 26.4 | 0.021 | |
| Global GAFamily=Qwen, n=32026.05 | 22.8 | 32.8 | 0.099 |