Knowledge Unlearning on WMDP bio (Quantization Metrics)
55.7BF16 ScoreZero-shot
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Zero-shotModel=Gemma-3-4B2026.05 | 55.7 | 55.7 | 0 | |
| Zero-shotModel=Gemma-3-1B2026.05 | 50 | 50 | 0 | |
| NPOModel=Gemma-3-4B2026.05 | 49.1 | 47.8 | 0.013 | |
| SimNPOModel=Gemma-3-4B2026.05 | 48.7 | 49.4 | 0.007 | |
| GU+SimNPOModel=Gemma-3-4B2026.05 | 48 | 48.7 | 0.007 | |
| Zero-shotModel=Gemma-2B2026.05 | 47 | 47 | 0 | |
| MANSUModel=Gemma-3-4B2026.05 | 36.4 | 33.7 | 0.027 | |
| NPOModel=Gemma-3-1B2026.05 | 34.3 | 30.7 | 0.037 | |
| SimNPOModel=Gemma-2B2026.05 | 34 | 33 | 0.01 | |
| GU+SimNPOModel=Gemma-2B2026.05 | 33.7 | 32.7 | 0.01 | |
| SimNPOModel=Gemma-3-1B2026.05 | 33.3 | 34 | 0.007 | |
| NPOModel=Gemma-2B2026.05 | 33 | 32.3 | 0.007 | |
| GU+SimNPOModel=Gemma-3-1B2026.05 | 31.3 | 32.7 | 0.013 | |
| MANSUModel=Gemma-3-1B2026.05 | 30 | 27.7 | 0.023 | |
| MANSUModel=Gemma-2B2026.05 | 28.4 | 26.4 | 0.02 |