Multi-subject Reasoning on MMLU-Pro
65.6Acc (Mean)Gemma-3-12B-Instruct
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Gemma-3-12B-InstructSetting=Optimized2025.12 | 65.6 | 0.0128 | 0.333 | 102,069.6 | |
| Gemma-3-12B-InstructSetting=Random2025.12 | 50.1 | 0.008 | 0.263 | 114,524.53 | |
| Qwen2.5-7B-InstructSetting=Optimized2025.12 | 49.7 | 0.018 | 0.174 | 128,362.8 | |
| Qwen2.5-7B-InstructSetting=Random2025.12 | 39.9 | 0.015 | 0.114 | 372,446.18 | |
| Llama-3.1-8B-InstructSetting=Optimized2025.12 | 37.3 | 0.011 | 0.155 | 513,244.2 | |
| Llama-3.1-8B-InstructSetting=Random2025.12 | 31.9 | 0.01 | 0.109 | 822,103.05 |