Mathematical Reasoning on Beyond-AIME v1 (test)
76.6Avg@5Gemini-3-Pro-Preview
Evaluation results
| Method | Links | |
|---|---|---|
| Gemini-3-Pro-PreviewEvaluation Protocol=seeds, Sample Size=1002026.01 | 76.6 | |
| Gemini-3-Pro-PreviewEvaluation Protocol=VeRA_E, Sample Size=1002026.01 | 74 | |
| GPT-5.1-highEvaluation Protocol=VeRA_E, Sample Size=1002026.01 | 73 | |
| DeepSeek-V3.1-thinkingEvaluation Protocol=seeds, Sample Size=1002026.01 | 71.8 | |
| GPT-5.1-highEvaluation Protocol=seeds, Sample Size=1002026.01 | 71.6 | |
| GPT-5-highEvaluation Protocol=VeRA_E, Sample Size=1002026.01 | 69.8 | |
| DeepSeek-V3.2-thinkingEvaluation Protocol=seeds, Sample Size=1002026.01 | 69.4 | |
| GLM-4.6Evaluation Protocol=seeds, Sample Size=1002026.01 | 68.2 | |
| GPT-5-highEvaluation Protocol=seeds, Sample Size=1002026.01 | 68 | |
| DeepSeek-V3.2-thinkingEvaluation Protocol=VeRA_E, Sample Size=1002026.01 | 67.6 | |
| GLM-4.6Evaluation Protocol=VeRA_E, Sample Size=1002026.01 | 66.4 | |
| DeepSeek-V3.1-thinkingEvaluation Protocol=VeRA_E, Sample Size=1002026.01 | 66 | |
| Seed-1.6-1015-highEvaluation Protocol=seeds, Sample Size=1002026.01 | 58.8 | |
| qwen3-max-0923Evaluation Protocol=VeRA_E, Sample Size=1002026.01 | 58.8 | |
| Seed-1.6-1015-highEvaluation Protocol=VeRA_E, Sample Size=1002026.01 | 58.2 | |
| Gemini-2.5-ProEvaluation Protocol=seeds, Sample Size=1002026.01 | 57.2 | |
| Gemini-2.5-ProEvaluation Protocol=VeRA_E, Sample Size=1002026.01 | 57 | |
| Minimax-M2Evaluation Protocol=seeds, Sample Size=1002026.01 | 56.8 | |
| Seed-1.6-Thinking-0715Evaluation Protocol=VeRA_E, Sample Size=1002026.01 | 55.8 | |
| Kimi-K2-thinkingEvaluation Protocol=seeds, Sample Size=1002026.01 | 54.8 | |
| Seed-1.6-Lite-1015-highEvaluation Protocol=seeds, Sample Size=1002026.01 | 54.8 | |
| Seed-1.6-Thinking-0715Evaluation Protocol=seeds, Sample Size=1002026.01 | 54.4 | |
| Minimax-M2Evaluation Protocol=VeRA_E, Sample Size=1002026.01 | 54.4 | |
| qwen3-max-0923Evaluation Protocol=seeds, Sample Size=1002026.01 | 53.6 | |
| Seed-1.6-Lite-1015-highEvaluation Protocol=VeRA_E, Sample Size=1002026.01 | 53.6 | |
| Claude-Sonnet-4.5-thinkingEvaluation Protocol=VeRA_E, Sample Size=1002026.01 | 53 | |
| Claude-Sonnet-4.5-thinkingEvaluation Protocol=seeds, Sample Size=1002026.01 | 51.6 | |
| Kimi-K2-thinkingEvaluation Protocol=VeRA_E, Sample Size=1002026.01 | 44 | |
| Kimi-K2-0905Evaluation Protocol=seeds, Sample Size=1002026.01 | 37 | |
| Kimi-K2-0905Evaluation Protocol=VeRA_E, Sample Size=1002026.01 | 34.6 | |
| GPT-5.1-chat-latestEvaluation Protocol=VeRA_E, Sample Size=1002026.01 | 30.6 | |
| GPT-5.1-chat-latestEvaluation Protocol=seeds, Sample Size=1002026.01 | 28.8 |