Mathematical Reasoning on GSM8K (Avg@5)
97.1Avg@5 Success RateGPT-5-high
Evaluation results
| Method | Links | |
|---|---|---|
| GPT-5-highEvaluation Variant=VeRA-E, Sample Count=26382026.01 | 97.1 | |
| GPT-5.1-highEvaluation Variant=VeRA-E, Sample Count=26382026.01 | 96.9 | |
| qwen3-max-0923Evaluation Variant=VeRA-E, Sample Count=26382026.01 | 96.9 | |
| Seed-1.6-1015-highEvaluation Variant=VeRA-E, Sample Count=26382026.01 | 96.3 | |
| Claude-Sonnet-4.5-thinkingEvaluation Variant=VeRA-E, Sample Count=26382026.01 | 96.2 | |
| GPT-5.1-chat-latestEvaluation Variant=VeRA-E, Sample Count=26382026.01 | 96 | |
| Seed-1.6-Thinking-0715Evaluation Variant=VeRA-E, Sample Count=26382026.01 | 95.9 | |
| Seed-1.6-Lite-1015-highEvaluation Variant=VeRA-E, Sample Count=26382026.01 | 95.8 | |
| GPT-5.1-highEvaluation Variant=seeds, Sample Count=13192026.01 | 95.6 | |
| Seed-1.6-1015-highEvaluation Variant=seeds, Sample Count=13192026.01 | 95.6 | |
| GPT-5-highEvaluation Variant=seeds, Sample Count=13192026.01 | 95.6 | |
| Kimi-K2-0905Evaluation Variant=VeRA-E, Sample Count=26382026.01 | 95.6 | |
| qwen3-max-0923Evaluation Variant=seeds, Sample Count=13192026.01 | 95.5 | |
| DeepSeek-V3.2-thinkingEvaluation Variant=VeRA-E, Sample Count=26382026.01 | 95.4 | |
| Claude-Sonnet-4.5-thinkingEvaluation Variant=seeds, Sample Count=13192026.01 | 95.3 | |
| Seed-1.6-Thinking-0715Evaluation Variant=seeds, Sample Count=13192026.01 | 95.1 | |
| Seed-1.6-Lite-1015-highEvaluation Variant=seeds, Sample Count=13192026.01 | 95.1 | |
| Gemini-2.5-ProEvaluation Variant=VeRA-E, Sample Count=26382026.01 | 95 | |
| Kimi-K2-thinkingEvaluation Variant=seeds, Sample Count=13192026.01 | 95 | |
| Gemini-3-Pro-PreviewEvaluation Variant=VeRA-E, Sample Count=26382026.01 | 95 | |
| Kimi-K2-0905Evaluation Variant=seeds, Sample Count=13192026.01 | 95 | |
| Minimax-M2Evaluation Variant=VeRA-E, Sample Count=26382026.01 | 94.9 | |
| GLM-4.6Evaluation Variant=VeRA-E, Sample Count=26382026.01 | 94.8 | |
| DeepSeek-V3.1-thinkingEvaluation Variant=VeRA-E, Sample Count=26382026.01 | 94.8 | |
| Minimax-M2Evaluation Variant=seeds, Sample Count=13192026.01 | 94.7 | |
| Gemini-3-Pro-PreviewEvaluation Variant=seeds, Sample Count=13192026.01 | 94.5 | |
| DeepSeek-V3.1-thinkingEvaluation Variant=seeds, Sample Count=13192026.01 | 94.5 | |
| GPT-5.1-chat-latestEvaluation Variant=seeds, Sample Count=13192026.01 | 94.3 | |
| GLM-4.6Evaluation Variant=seeds, Sample Count=13192026.01 | 94.1 | |
| DeepSeek-V3.2-thinkingEvaluation Variant=seeds, Sample Count=13192026.01 | 94 | |
| Gemini-2.5-ProEvaluation Variant=seeds, Sample Count=13192026.01 | 93.9 | |
| Kimi-K2-thinkingEvaluation Variant=VeRA-E, Sample Count=26382026.01 | 86.6 |