Utility Evaluation on MMLU (MMLU Score)
49.5MMLU ScoreZero-shot
Evaluation Results
| Method | Links | |
|---|---|---|
| Zero-shotModel=Gemma-3-4B, Unlearning Domain=bio2026.05 | 49.5 | |
| Zero-shotModel=Gemma-3-4B, Unlearning Domain=chem2026.05 | 49.5 | |
| Zero-shotModel=Gemma-3-4B, Unlearning Domain=cyber2026.05 | 49.5 | |
| MANSUModel=Gemma-3-4B, Unlearning Domain=bio2026.05 | 47.8 | |
| MANSUModel=Gemma-3-4B, Unlearning Domain=cyber2026.05 | 47.8 | |
| MANSUModel=Gemma-3-4B, Unlearning Domain=chem2026.05 | 47.4 | |
| GU+SimNPOModel=Gemma-3-4B, Unlearning Domain=bio2026.05 | 47.2 | |
| NPOModel=Gemma-3-4B, Unlearning Domain=bio2026.05 | 47.1 | |
| SimNPOModel=Gemma-3-4B, Unlearning Domain=bio2026.05 | 47.1 | |
| GU+SimNPOModel=Gemma-3-4B, Unlearning Domain=cyber2026.05 | 46.8 | |
| SimNPOModel=Gemma-3-4B, Unlearning Domain=cyber2026.05 | 46.8 | |
| NPOModel=Gemma-3-4B, Unlearning Domain=chem2026.05 | 46.2 | |
| GU+SimNPOModel=Gemma-3-4B, Unlearning Domain=chem2026.05 | 46.1 | |
| SimNPOModel=Gemma-3-4B, Unlearning Domain=chem2026.05 | 46.1 | |
| NPOModel=Gemma-3-4B, Unlearning Domain=cyber2026.05 | 45.8 | |
| Zero-shotModel=Gemma-3-1B, Unlearning Domain=bio2026.05 | 41 | |
| Zero-shotModel=Gemma-3-1B, Unlearning Domain=chem2026.05 | 41 | |
| Zero-shotModel=Gemma-3-1B, Unlearning Domain=cyber2026.05 | 41 | |
| Zero-shotModel=Gemma-2B, Unlearning Domain=bio2026.05 | 40.5 | |
| Zero-shotModel=Gemma-2B, Unlearning Domain=chem2026.05 | 40.5 | |
| Zero-shotModel=Gemma-2B, Unlearning Domain=cyber2026.05 | 40.5 | |
| MANSUModel=Gemma-3-1B, Unlearning Domain=bio2026.05 | 39.7 | |
| MANSUModel=Gemma-3-1B, Unlearning Domain=cyber2026.05 | 39.7 | |
| MANSUModel=Gemma-3-1B, Unlearning Domain=chem2026.05 | 39.4 | |
| MANSUModel=Gemma-2B, Unlearning Domain=bio2026.05 | 39.1 | |
| MANSUModel=Gemma-2B, Unlearning Domain=cyber2026.05 | 39.1 | |
| MANSUModel=Gemma-2B, Unlearning Domain=chem2026.05 | 38.8 | |
| NPOModel=Gemma-2B, Unlearning Domain=chem2026.05 | 37 | |
| NPOModel=Gemma-2B, Unlearning Domain=bio2026.05 | 35.5 | |
| NPOModel=Gemma-2B, Unlearning Domain=cyber2026.05 | 28.3 | |
| GU+SimNPOModel=Gemma-2B, Unlearning Domain=bio2026.05 | 27.5 | |
| SimNPOModel=Gemma-3-1B, Unlearning Domain=cyber2026.05 | 27.5 | |
| SimNPOModel=Gemma-2B, Unlearning Domain=bio2026.05 | 27 | |
| NPOModel=Gemma-3-1B, Unlearning Domain=chem2026.05 | 26.5 | |
| GU+SimNPOModel=Gemma-3-1B, Unlearning Domain=bio2026.05 | 26.3 | |
| GU+SimNPOModel=Gemma-3-1B, Unlearning Domain=cyber2026.05 | 26.3 | |
| SimNPOModel=Gemma-3-1B, Unlearning Domain=chem2026.05 | 25.8 | |
| GU+SimNPOModel=Gemma-2B, Unlearning Domain=chem2026.05 | 25.5 | |
| SimNPOModel=Gemma-2B, Unlearning Domain=chem2026.05 | 25.5 | |
| NPOModel=Gemma-3-1B, Unlearning Domain=bio2026.05 | 25.3 | |
| SimNPOModel=Gemma-3-1B, Unlearning Domain=bio2026.05 | 25.3 | |
| GU+SimNPOModel=Gemma-3-1B, Unlearning Domain=chem2026.05 | 24.8 | |
| GU+SimNPOModel=Gemma-2B, Unlearning Domain=cyber2026.05 | 20.8 | |
| SimNPOModel=Gemma-2B, Unlearning Domain=cyber2026.05 | 20.8 | |
| NPOModel=Gemma-3-1B, Unlearning Domain=cyber2026.05 | 20.3 |