Question Answering on GPQA Diamond v1 (test)
86.7Avg@5Gemini-3-Pro-Preview
Evaluation results
| Method | Links | |
|---|---|---|
| Gemini-3-Pro-PreviewEvaluation Protocol=seeds, Sample Size=1982026.01 | 86.7 | |
| GPT-5.1-highEvaluation Protocol=seeds, Sample Size=1982026.01 | 86.4 | |
| Gemini-3-Pro-PreviewEvaluation Protocol=VeRA_E, Sample Size=8712026.01 | 85.7 | |
| GPT-5.1-highEvaluation Protocol=VeRA_E, Sample Size=8712026.01 | 84.5 | |
| Kimi-K2-thinkingEvaluation Protocol=VeRA_E, Sample Size=8712026.01 | 83.4 | |
| DeepSeek-V3.2-thinkingEvaluation Protocol=VeRA_E, Sample Size=8712026.01 | 83.3 | |
| Gemini-2.5-ProEvaluation Protocol=seeds, Sample Size=1982026.01 | 82.1 | |
| DeepSeek-V3.2-thinkingEvaluation Protocol=seeds, Sample Size=1982026.01 | 82.1 | |
| GPT-5-highEvaluation Protocol=VeRA_E, Sample Size=8712026.01 | 82.1 | |
| GPT-5-highEvaluation Protocol=seeds, Sample Size=1982026.01 | 81.9 | |
| Gemini-2.5-ProEvaluation Protocol=VeRA_E, Sample Size=8712026.01 | 81.4 | |
| DeepSeek-V3.1-thinkingEvaluation Protocol=VeRA_E, Sample Size=8712026.01 | 81.3 | |
| Claude-Sonnet-4.5-thinkingEvaluation Protocol=seeds, Sample Size=1982026.01 | 81 | |
| GLM-4.6Evaluation Protocol=VeRA_E, Sample Size=8712026.01 | 80.2 | |
| Claude-Sonnet-4.5-thinkingEvaluation Protocol=VeRA_E, Sample Size=8712026.01 | 80.1 | |
| DeepSeek-V3.1-thinkingEvaluation Protocol=seeds, Sample Size=1982026.01 | 80 | |
| Kimi-K2-thinkingEvaluation Protocol=seeds, Sample Size=1982026.01 | 79.9 | |
| Seed-1.6-1015-highEvaluation Protocol=VeRA_E, Sample Size=8712026.01 | 78.9 | |
| GLM-4.6Evaluation Protocol=seeds, Sample Size=1982026.01 | 78.8 | |
| Seed-1.6-Thinking-0715Evaluation Protocol=VeRA_E, Sample Size=8712026.01 | 78.5 | |
| Seed-1.6-Thinking-0715Evaluation Protocol=seeds, Sample Size=1982026.01 | 78.4 | |
| Seed-1.6-Lite-1015-highEvaluation Protocol=seeds, Sample Size=1982026.01 | 77.6 | |
| Seed-1.6-1015-highEvaluation Protocol=seeds, Sample Size=1982026.01 | 77.4 | |
| Seed-1.6-Lite-1015-highEvaluation Protocol=VeRA_E, Sample Size=8712026.01 | 76.9 | |
| Minimax-M2Evaluation Protocol=seeds, Sample Size=1982026.01 | 76.7 | |
| qwen3-max-0923Evaluation Protocol=VeRA_E, Sample Size=8712026.01 | 76.7 | |
| qwen3-max-0923Evaluation Protocol=seeds, Sample Size=1982026.01 | 76.1 | |
| Minimax-M2Evaluation Protocol=VeRA_E, Sample Size=8712026.01 | 74.4 | |
| Kimi-K2-0905Evaluation Protocol=VeRA_E, Sample Size=8712026.01 | 74.2 | |
| Kimi-K2-0905Evaluation Protocol=seeds, Sample Size=1982026.01 | 73.5 | |
| GPT-5.1-chat-latestEvaluation Protocol=seeds, Sample Size=1982026.01 | 69.9 | |
| GPT-5.1-chat-latestEvaluation Protocol=VeRA_E, Sample Size=8712026.01 | 69.1 |