Mathematics on MATH 500
97.3Pass@1DeepSeek-R1
Evaluation Results
| Method | Links | |
|---|---|---|
| DeepSeek-R1Architecture=MoE, Activated Params=37B, Total Params=671B2025.01 | 97.3 | |
| MiMo-RL 7B-R-TAPModel Backbone=7B2026.03 | 97.3 | |
| OpenAI-o1-12172025.01 | 96.4 | |
| MiMo 7B-RLModel Backbone=7B2026.03 | 95.8 | |
| R1-Distill-Qwen-14BModel Backbone=Qwen-14B2026.03 | 93.9 | |
| LearnedBackbone Model=DS-R1-Qwen-7B, Cmax=3, Bmax=92162025.06 | 92.89 | |
| R1-Distill-Qwen-7BModel Backbone=Qwen-7B2026.03 | 92.8 | |
| WaitBackbone Model=DS-R1-Qwen-7B, Cmax=3, Bmax=92162025.06 | 92.17 | |
| HmmBackbone Model=DS-R1-Qwen-7B, Cmax=3, Bmax=92162025.06 | 91.74 | |
| INVERSE-EVIDENCEModel=QWEN3-4B2026.03 | 91.23 | |
| RANDOMModel=QWEN3-4B2026.03 | 91.22 | |
| INSIGHTModel=QWEN3-4B2026.03 | 91.22 | |
| MoPPSModel=QWEN3-4B2026.03 | 90.82 | |
| DSModel=R1-DISTILL-QWEN-7B2026.03 | 90.82 | |
| QwQ-32B PreviewModel Backbone=QwQ-32B2026.03 | 90.6 | |
| RANDOMModel=R1-DISTILL-QWEN-7B2026.03 | 90.5 | |
| INSIGHTModel=R1-DISTILL-QWEN-7B2026.03 | 90.45 | |
| EXPECTED-DIFFICULTYModel=R1-DISTILL-QWEN-7B2026.03 | 90.43 | |
| MoPPSModel=R1-DISTILL-QWEN-7B2026.03 | 90.4 | |
| INVERSE-EVIDENCEModel=R1-DISTILL-QWEN-7B2026.03 | 90.4 | |
| DeepSeek-V3Architecture=MoE, Activated Params=37B, Total Params=671B2025.01 | 90.2 | |
| LearnedBackbone Model=DS-R1-Llama-8B, Cmax=3, Bmax=92162025.06 | 90.04 | |
| OpenAI-o1-mini2025.01 | 90 | |
| OpenAI o1-miniModel Backbone=o1-mini2026.03 | 90 | |
| AlternativelyBackbone Model=DS-R1-Qwen-7B, Cmax=3, Bmax=92162025.06 | 89.84 | |
| Qwen-3-30B-A3BOpenness=Open-weights, Regional Origin=Non-European, Post-training=Instruction-tuned2026.02 | 89.7 | |
| Baseline (w/o BF)Backbone Model=DS-R1-Qwen-7B, Cmax=3, Bmax=92162025.06 | 89.39 | |
| CritiqueBackbone Model=DS-R1-Qwen-7B, Cmax=3, Bmax=92162025.06 | 89.06 | |
| Gemma-3-27BOpenness=Open-weights, Regional Origin=Non-European, Post-training=Instruction-tuned2026.02 | 88.5 | |
| Qwen-3-14BOpenness=Open-weights, Regional Origin=Non-European, Post-training=Instruction-tuned2026.02 | 86.9 | |
| HmmBackbone Model=DS-R1-Llama-8B, Cmax=3, Bmax=92162025.06 | 86.21 | |
| WaitBackbone Model=DS-R1-Llama-8B, Cmax=3, Bmax=92162025.06 | 86.01 | |
| Scaf-GRPOBase Model=DeepSeek-R1-Distill-Qwen-1.5B2025.10 | 85.8 | |
| OLMo-3.1-32BOpenness=Fully-open, Regional Origin=Non-European, Post-training=Instruction-tuned2026.02 | 85.7 | |
| Qwen-3-32BOpenness=Open-weights, Regional Origin=Non-European, Post-training=Instruction-tuned2026.02 | 85.7 | |
| Gemma-3-12BOpenness=Open-weights, Regional Origin=Non-European, Post-training=Instruction-tuned2026.02 | 85.3 | |
| GT-RewardBase Model=Qwen3-8B-Base2025.11 | 84.72 | |
| OLMo-3-7BOpenness=Fully-open, Regional Origin=Non-European, Post-training=Instruction-tuned2026.02 | 84.2 | |
| DeepSeek-R1-Distill-Qwen-1.5BBase Model=DeepSeek-R1-Distill-Qwen-1.5B2025.10 | 83.9 | |
| Vanilla GRPOBase Model=DeepSeek-R1-Distill-Qwen-1.5B2025.10 | 83.9 | |
| GT-RewardBase Model=Qwen3-4B-Base2025.11 | 83.4 | |
| LearnedBackbone Model=DS-R1-Qwen-1.5B, Cmax=3, Bmax=92162025.06 | 82.51 | |
| AlternativelyBackbone Model=DS-R1-Llama-8B, Cmax=3, Bmax=92162025.06 | 81.86 | |
| Mistral-3.2-24BOpenness=Open-weights, Regional Origin=European, Post-training=Instruction-tuned2026.02 | 81.5 | |
| WaitBackbone Model=DS-R1-Qwen-1.5B, Cmax=3, Bmax=92162025.06 | 81.22 | |
| HmmBackbone Model=DS-R1-Qwen-1.5B, Cmax=3, Bmax=92162025.06 | 80.84 | |
| AlternativelyBackbone Model=DS-R1-Qwen-1.5B, Cmax=3, Bmax=92162025.06 | 80.33 | |
| CritiqueBackbone Model=DS-R1-Llama-8B, Cmax=3, Bmax=92162025.06 | 80.27 | |
| Scaf-GRPOBase Model=Qwen2.5-Math-7B2025.10 | 80 | |
| Self-HarmonyBase Model=Qwen3-8B-Base2025.11 | 80 | |
| Baseline (w/o BF)Backbone Model=DS-R1-Qwen-1.5B, Cmax=3, Bmax=92162025.06 | 79.88 | |
| Baseline (w/o BF)Backbone Model=DS-R1-Llama-8B, Cmax=3, Bmax=92162025.06 | 79.56 | |
| CritiqueBackbone Model=DS-R1-Qwen-1.5B, Cmax=3, Bmax=92162025.06 | 79.49 | |
| Majority-VotingBase Model=Qwen3-8B-Base2025.11 | 78.99 | |
| Co-RewardBase Model=Qwen3-8B-Base2025.11 | 78.92 | |
| IntuitorBase Model=Qwen3-8B-Base2025.11 | 78.77 | |
| Self-HarmonyBase Model=Qwen3-4B-Base2025.11 | 78.5 | |
| Claude-3.5-Sonnet-10222025.01 | 78.3 | |
| Claude 3.5 Sonnet-1022Model Backbone=Claude 3.5 Sonnet2026.03 | 78.3 | |
| Oat-ZeroBase Model=Qwen2.5-Math-7B2025.10 | 78 | |
| Scaf-GRPOBase Model=Qwen2.5-7B2025.10 | 77.8 | |
| Vanilla GRPOBase Model=Qwen2.5-7B2025.10 | 77.6 | |
| RentBase Model=Qwen3-8B-Base2025.11 | 77.26 | |
| SimpleRL-ZeroBase Model=Qwen2.5-Math-7B2025.10 | 76.8 | |
| Co-RewardBase Model=Qwen3-4B-Base2025.11 | 76.54 | |
| Vanilla GRPOBase Model=Qwen2.5-Math-7B2025.10 | 75.8 | |
| Majority-VotingBase Model=Qwen3-4B-Base2025.11 | 75.75 | |
| LUFFYBase Model=Qwen2.5-Math-7B2025.10 | 75.2 | |
| GPT-4o-05132025.01 | 74.6 | |
| Llama-3.3-70BOpenness=Open-weights, Regional Origin=Non-European, Post-training=Instruction-tuned2026.02 | 74.6 | |
| RentBase Model=Qwen3-4B-Base2025.11 | 74.6 | |
| GPT-4o 0513Model Backbone=GPT-4o2026.03 | 74.6 | |
| GT-RewardBase Model=Llama-3.1-8B-Instruct2025.11 | 73.6 | |
| Scaf-GRPOBase Model=Qwen2.5-Math-1.5B2025.10 | 73.4 | |
| IntuitorBase Model=Qwen3-4B-Base2025.11 | 72.35 | |
| Vanilla GRPOBase Model=Qwen2.5-Math-1.5B2025.10 | 72.2 | |
| GT-RewardBase Model=Qwen3-1.7B-Base2025.11 | 71.8 | |
| INVERSE-EVIDENCEModel=QWEN3-0.6B2026.03 | 71.8 | |
| INSIGHTModel=QWEN3-0.6B2026.03 | 71.75 | |
| RANDOMModel=QWEN3-0.6B2026.03 | 71.3 | |
| MoPPSModel=QWEN3-0.6B2026.03 | 70.6 | |
| GT-RewardBase Model=Llama-3.2-3B-Instruct2025.11 | 69.8 | |
| Self-HarmonyBase Model=Qwen3-1.7B-Base2025.11 | 69.6 | |
| Before RLBase Model=Qwen3-8B-Base2025.11 | 66.8 | |
| Co-RewardBase Model=Qwen3-1.7B-Base2025.11 | 64.67 | |
| Majority-VotingBase Model=Qwen3-1.7B-Base2025.11 | 64.64 | |
| Qwen2.5-7BBase Model=Qwen2.5-7B2025.10 | 61.8 | |
| RentBase Model=Qwen3-1.7B-Base2025.11 | 61.08 | |
| Before RLBase Model=Qwen3-4B-Base2025.11 | 60.2 | |
| Scaf-GRPOBase Model=Llama-3.2-3B-Instruct2025.10 | 56.2 | |
| Self-HarmonyBase Model=Llama-3.2-3B-Instruct2025.11 | 55.4 | |
| Co-RewardBase Model=Llama-3.2-3B-Instruct2025.11 | 55.22 | |
| EuroLLM-22BOpenness=Fully-open, Regional Origin=European, Post-training=Instruction-tuned2026.02 | 54.5 | |
| Qwen2.5-Math-7BBase Model=Qwen2.5-Math-7B2025.10 | 53.6 | |
| Vanilla GRPOBase Model=Llama-3.2-3B-Instruct2025.10 | 51.8 | |
| IntuitorBase Model=Qwen3-1.7B-Base2025.11 | 51.12 | |
| Self-HarmonyBase Model=Llama-3.1-8B-Instruct2025.11 | 50.4 | |
| Llama-3.1-8BOpenness=Open-weights, Regional Origin=Non-European, Post-training=Instruction-tuned2026.02 | 49.4 | |
| IntuitorBase Model=Llama-3.1-8B-Instruct2025.11 | 48.37 | |
| Co-RewardBase Model=Llama-3.1-8B-Instruct2025.11 | 48.01 |