Mathematical Reasoning on BRUMO 2025 (PASS@1, R, Delta)
69.48PASS@1QUESTA-Nemotron-1.5B
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| QUESTA-Nemotron-1.5Bk (responses per question)=32, sampling temperature=0.7, top-p=0.952025.07 | 69.48 | — | — | |
| Qwen3-8Bk (responses per question)=32, sampling temperature=0.7, top-p=0.952025.07 | 68.33 | — | — | |
| DeepSeek-R1-Distill-32Bk (responses per question)=32, sampling temperature=0.7, top-p=0.952025.07 | 68 | — | — | |
| Gemini-2.5-FModel=Gemini-2.5-Flash2025.10 | 63.4 | 20.7 | 42.7 | |
| Nemotron-1.5Bk (responses per question)=32, sampling temperature=0.7, top-p=0.952025.07 | 58.23 | — | — | |
| Qwen3-30BModel=Qwen3-30B2025.10 | 46.8 | 21.8 | 25 | |
| Qwen3-1.7Bk (responses per question)=32, sampling temperature=0.7, top-p=0.952025.07 | 44.06 | — | — | |
| Gemini-2.5-F-LModel=Gemini-2.5-Flash-Lite2025.10 | 43.2 | 17.8 | 25.4 | |
| GPT-4.1-MModel=GPT-4.1-mini2025.10 | 34.4 | 24.6 | 9.8 | |
| GPT-4.1Model=GPT-4.12025.10 | 33.3 | 22.8 | 10.5 | |
| DeepSeek-R1-Distill-1.5Bk (responses per question)=32, sampling temperature=0.7, top-p=0.952025.07 | 31.8 | — | — |