Mathematical Reasoning on AIME 2024 (Avg@5)
94Avg@5 ScoreGemini-3-Pro-Preview
Evaluation results
| Method | Links | |
|---|---|---|
| Gemini-3-Pro-PreviewEvaluation Variant=seeds_E, Sample Count=302026.01 | 94 | |
| GPT-5.1-highEvaluation Variant=seeds_E, Sample Count=302026.01 | 92 | |
| DeepSeek-V3.2-thinkingEvaluation Variant=seeds_E, Sample Count=302026.01 | 92 | |
| GLM-4.6Evaluation Variant=seeds_E, Sample Count=302026.01 | 90.7 | |
| DeepSeek-V3.1-thinkingEvaluation Variant=seeds_E, Sample Count=302026.01 | 90.7 | |
| GPT-5-highEvaluation Variant=seeds_E, Sample Count=302026.01 | 90 | |
| Seed-1.6-1015-highEvaluation Variant=seeds_E, Sample Count=302026.01 | 89.3 | |
| Gemini-2.5-ProEvaluation Variant=seeds_E, Sample Count=302026.01 | 88 | |
| Seed-1.6-Lite-1015-highEvaluation Variant=seeds_E, Sample Count=302026.01 | 87.3 | |
| Kimi-K2-thinkingEvaluation Variant=seeds_E, Sample Count=302026.01 | 86.7 | |
| Seed-1.6-Thinking-0715Evaluation Variant=seeds_E, Sample Count=302026.01 | 85.3 | |
| Minimax-M2Evaluation Variant=seeds_E, Sample Count=302026.01 | 84.7 | |
| Claude-Sonnet-4.5-thinkingEvaluation Variant=seeds_E, Sample Count=302026.01 | 84 | |
| qwen3-max-0923Evaluation Variant=seeds_E, Sample Count=302026.01 | 82.7 | |
| Gemini-3-Pro-PreviewEvaluation Variant=VeRA-E, Sample Count=602026.01 | 81.3 | |
| DeepSeek-V3.2-thinkingEvaluation Variant=VeRA-E, Sample Count=602026.01 | 79 | |
| DeepSeek-V3.1-thinkingEvaluation Variant=VeRA-E, Sample Count=602026.01 | 77.7 | |
| GPT-5.1-highEvaluation Variant=VeRA-E, Sample Count=602026.01 | 75.3 | |
| Kimi-K2-thinkingEvaluation Variant=VeRA-E, Sample Count=602026.01 | 75.3 | |
| Seed-1.6-1015-highEvaluation Variant=VeRA-E, Sample Count=602026.01 | 75 | |
| GPT-5-highEvaluation Variant=VeRA-E, Sample Count=602026.01 | 74.7 | |
| Claude-Sonnet-4.5-thinkingEvaluation Variant=VeRA-E, Sample Count=602026.01 | 74 | |
| Gemini-2.5-ProEvaluation Variant=VeRA-E, Sample Count=602026.01 | 73.3 | |
| Seed-1.6-Lite-1015-highEvaluation Variant=VeRA-E, Sample Count=602026.01 | 73 | |
| Seed-1.6-Thinking-0715Evaluation Variant=VeRA-E, Sample Count=602026.01 | 72 | |
| GLM-4.6Evaluation Variant=VeRA-E, Sample Count=602026.01 | 71.7 | |
| qwen3-max-0923Evaluation Variant=VeRA-E, Sample Count=602026.01 | 68.7 | |
| Kimi-K2-0905Evaluation Variant=seeds_E, Sample Count=302026.01 | 65.3 | |
| Minimax-M2Evaluation Variant=VeRA-E, Sample Count=602026.01 | 55.7 | |
| Kimi-K2-0905Evaluation Variant=VeRA-E, Sample Count=602026.01 | 51.3 | |
| GPT-5.1-chat-latestEvaluation Variant=seeds_E, Sample Count=302026.01 | 48.7 | |
| GPT-5.1-chat-latestEvaluation Variant=VeRA-E, Sample Count=602026.01 | 46 |