Mathematical Reasoning on AIME 2024 (Pass@1, Avg@8, Pass@8)
93.4Pass@1o4-mini (high)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| o4-mini (high)# Params=–, Scale Category=Large Scale (> 70B)2026.03 | 93.4 | — | — | |
| DeepSeek-R1-0528# Params=671B, Scale Category=Large Scale (> 70B)2026.03 | 91.4 | — | — | |
| Qwen3-4B-Thinking-2507 + CHIMERA# Params=4B, Scale Category=Small to Medium Scale (≤ 70B)2026.03 | 86.9 | — | — | |
| Qwen3-235B-A22B# Params=235B, Scale Category=Large Scale (> 70B)2026.03 | 85.7 | — | — | |
| DeepSeek-R1-0528-Qwen3-8B# Params=8B, Scale Category=Small to Medium Scale (≤ 70B)2026.03 | 82.2 | — | — | |
| Qwen3-4B-Thinking-2507# Params=4B, Scale Category=Small to Medium Scale (≤ 70B)2026.03 | 81.6 | — | — | |
| Qwen3-32B# Params=32B, Scale Category=Small to Medium Scale (≤ 70B)2026.03 | 81.4 | — | — | |
| DeepSeek-R1# Params=671B, Scale Category=Large Scale (> 70B)2026.03 | 79.8 | — | — | |
| o3-mini (medium)# Params=–, Scale Category=Large Scale (> 70B)2026.03 | 79.6 | — | — | |
| DeepSeek-R1-Distill-Llama-70B# Params=70B, Scale Category=Small to Medium Scale (≤ 70B)2026.03 | 70 | — | — | |
| Qwen3-4B-Thinking-2507 + OpenScience# Params=4B, Scale Category=Small to Medium Scale (≤ 70B)2026.03 | 61.7 | — | — | |
| MELBackbone=Qwen3-14B-Base2026.02 | 33.33 | 35.83 | 60 | |
| MELBackbone=Qwen3-8B-Base2026.02 | 30 | 25.42 | 60 | |
| GRPOBackbone=Qwen3-14B-Base2026.02 | 30 | 35.41 | 56.67 | |
| DeepCompress-Zero-7BParameters=7B, Training=DeepCompress2025.10 | 23.5 | — | — | |
| MELBackbone=Qwen3-4B-Base2026.02 | 20 | 20.83 | 33 | |
| DeepMath-Zero-7BParameters=7B, Training=Zero RL2025.10 | 19.4 | — | — | |
| DeepCompress-Zero-3BParameters=3B, Training=DeepCompress2025.10 | 16.7 | — | — | |
| GRPOBackbone=Qwen3-8B-Base2026.02 | 16.67 | 24.58 | 43.33 | |
| Open-Reasoner-Zero-7BParameters=7B, Training=Zero RL2025.10 | 15.6 | — | — | |
| Qwen-2.5-7B-SRL-ZooParameters=7B, Variant=SRL-Zoo2025.10 | 15.6 | — | — | |
| BaselineBackbone=Qwen3-4B-Base2026.02 | 13.33 | 9.9 | 30 | |
| GRPOBackbone=Qwen3-4B-Base2026.02 | 13.33 | 18.33 | 30 | |
| BaselineBackbone=Qwen3-14B-Base2026.02 | 13.33 | 10.83 | 36.67 | |
| DeepMath-Zero-3BParameters=3B, Training=Zero RL2025.10 | 11.5 | — | — | |
| Qwen-2.5-7BParameters=7B2025.10 | 7.7 | — | — | |
| BaselineBackbone=Qwen3-8B-Base2026.02 | 6.67 | 10 | 26.67 | |
| Qwen-2.5-3B-InstructParameters=3B, Variant=Instruct2025.10 | 5.4 | — | — | |
| Qwen-2.5-3BParameters=3B2025.10 | 4.2 | — | — |