Knowledge Unlearning on WMDP cyber (Quantization Metrics)
41BF16 Precision ScoreZero-shot
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Zero-shotModel=Gemma-3-4B2026.05 | 41 | 41 | 0 | |
| Zero-shotModel=Gemma-2B2026.05 | 38 | 38 | 0 | |
| NPOModel=Gemma-3-4B2026.05 | 36.1 | 35.4 | 0.007 | |
| SimNPOModel=Gemma-3-4B2026.05 | 36.1 | 36.4 | 0.003 | |
| GU+SimNPOModel=Gemma-3-4B2026.05 | 35.8 | 36.1 | 0.003 | |
| Zero-shotModel=Gemma-3-1B2026.05 | 33.7 | 33.7 | 0 | |
| NPOModel=Gemma-2B2026.05 | 31 | 29 | 0.02 | |
| GU+SimNPOModel=Gemma-3-1B2026.05 | 29.3 | 28.7 | 0.007 | |
| SimNPOModel=Gemma-3-1B2026.05 | 29 | 29.3 | 0.003 | |
| GU+SimNPOModel=Gemma-2B2026.05 | 28.7 | 29 | 0.003 | |
| SimNPOModel=Gemma-2B2026.05 | 28.7 | 28.7 | 0 | |
| NPOModel=Gemma-3-1B2026.05 | 28 | 27.3 | 0.007 | |
| MANSUModel=Gemma-3-4B2026.05 | 25.4 | 23.4 | 0.02 | |
| MANSUModel=Gemma-2B2026.05 | 23.4 | 21.7 | 0.017 | |
| MANSUModel=Gemma-3-1B2026.05 | 22.7 | 21 | 0.017 |