Knowledge Unlearning on WMDP chem
42.3BF16 ScoreZero-shot
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Zero-shotModel=Gemma-3-4B2026.05 | 42.3 | 42.3 | 0 | |
| Zero-shotModel=Gemma-2B2026.05 | 38.3 | 38.3 | 0 | |
| NPOModel=Gemma-3-4B2026.05 | 38.1 | 37.4 | 0.007 | |
| SimNPOModel=Gemma-3-4B2026.05 | 37.4 | 38.1 | 0.007 | |
| GU+SimNPOModel=Gemma-3-4B2026.05 | 37.1 | 37.8 | 0.007 | |
| Zero-shotModel=Gemma-3-1B2026.05 | 33.7 | 33.7 | 0 | |
| NPOModel=Gemma-2B2026.05 | 29.7 | 29.3 | 0.003 | |
| SimNPOModel=Gemma-2B2026.05 | 27 | 27.7 | 0.007 | |
| MANSUModel=Gemma-3-4B2026.05 | 27 | 24.7 | 0.023 | |
| GU+SimNPOModel=Gemma-2B2026.05 | 26.3 | 28.3 | 0.02 | |
| SimNPOModel=Gemma-3-1B2026.05 | 25 | 23.3 | 0.017 | |
| NPOModel=Gemma-3-1B2026.05 | 24.7 | 25 | 0.003 | |
| GU+SimNPOModel=Gemma-3-1B2026.05 | 24.3 | 24.7 | 0.003 | |
| MANSUModel=Gemma-2B2026.05 | 22.4 | 20.7 | 0.017 | |
| MANSUModel=Gemma-3-1B2026.05 | 20.7 | 19 | 0.017 |