Mathematical Reasoning on PyraMathBench text-only 1.0 (test)
93.1MWP AccuracyDeepSeek-R1
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| DeepSeek-R1Size=671B2026.06 | 93.1 | 87.7 | 75.6 | 100 | 96.2 | 99.5 | 97.4 | 100 | 81.9 | 78.6 | |
| GPT-4oSize=-2026.06 | 92.1 | 66.1 | 66.6 | 75.6 | 96 | 97.5 | 95.8 | 76.7 | 83.8 | 69.7 | |
| Claude-opus-4.6Size=-2026.06 | 91.8 | 87.4 | 97.8 | 84.6 | 95.1 | 99.1 | 94.7 | 86.7 | 69.8 | 82.7 | |
| Qwen-2.5Size=14B2026.06 | 91.1 | 64.5 | 62.7 | 83.5 | 90.4 | 98.6 | 96 | 79.4 | 80.9 | 36.6 | |
| GPT-4o miniSize=-2026.06 | 89.6 | 64.7 | 61.2 | 80.8 | 95.7 | 98.3 | 96.6 | 60.5 | 72.8 | 73.5 | |
| Gemma2Size=9B2026.06 | 87.4 | 53.1 | 77.7 | 78.3 | 81.4 | 95.6 | 93.9 | 71.7 | 69 | 14.9 | |
| Qwen-2.5-MathSize=7B2026.06 | 82.5 | 48.4 | 45.9 | 19.2 | 95.1 | 97.4 | 91.5 | 77.3 | 9.7 | 3.5 | |
| Llama3.1Size=8B2026.06 | 72.3 | 54.1 | 76.7 | 55.2 | 89.3 | 86.8 | 81.2 | 59.5 | 72.7 | 24.3 | |
| DeepSeek-MathSize=7B2026.06 | 59.3 | 34.7 | 54.7 | 40.1 | 84.8 | 90.6 | 64.6 | 21.5 | 74.9 | 55.4 | |
| MistralSize=7B2026.06 | 42.3 | 6.6 | 78.8 | 71 | 66.5 | 21.9 | 90.4 | 51.7 | 64.3 | 53.9 | |
| LLaVASize=13B2026.06 | 34.4 | 11.4 | 36.1 | 61 | 60.9 | 21.6 | 55.2 | 34.6 | 65.1 | 29.5 |