Mathematical Reasoning on MATH 500 (Accuracy Variance, Consistency, Token Variance)
79.7Mean AccuracyGemma-3-12B-Instruct
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Gemma-3-12B-InstructSetting=Optimized2025.12 | 79.7 | 0.0016 | 55.4 | 144,471.24 | |
| Qwen2.5-7B-InstructSetting=Optimized2025.12 | 68.6 | 0.007 | 35.4 | 122,866.7 | |
| Gemma-3-12B-InstructSetting=Random2025.12 | 67 | 0.002 | 49.5 | 153,144.84 | |
| Qwen2.5-7B-InstructSetting=Random2025.12 | 58.5 | 0.007 | 35.4 | 305,133.92 | |
| Llama-3.1-8B-InstructSetting=Optimized2025.12 | 28.5 | 0.026 | 1.1 | 1,547,849 | |
| Llama-3.1-8B-InstructSetting=Random2025.12 | 24.9 | 0.015 | 2.7 | 1,512,173.79 |