Mathematical Reasoning on OlympiadBench (accuracy)
0.7659AccuracyJustRL-Nemotron
Evaluation results
| Method | Links | ||
|---|---|---|---|
| JustRL-NemotronBackbone=OpenMath-Nemotron-1.5B, Sampling Strategy (@k)=@4, Dynamic Sampling=false, Training Steps=3440, Train Batch Size=256, Rollout N=8, Max Context Length=16k, Token Budget=1.1x10^8k2025.12 | 0.7659 | — | |
| QuestABackbone=OpenMath-Nemotron-1.5B, Sampling Strategy (@k)=@4, Dynamic Sampling=true, Training Steps=2000, Train Batch Size=128, Rollout N=16, Max Context Length=32k, Token Budget=2.6x10^8k2025.12 | 0.7228 | — | |
| BackboneBackbone=OpenMath-Nemotron-1.5B, Sampling Strategy (@k)=@42025.12 | 0.717 | — | |
| PMPO-7B (Ours) [R1-Distill]Backbone=DeepSeek-R1-Distill-Qwen-7B, Optimization Algorithm=PMPO2026.01 | 0.642 | — | |
| GMPO-7B [R1-Distill]Backbone=DeepSeek-R1-Distill-Qwen-7B, Optimization Algorithm=GMPO2026.01 | 0.625 | — | |
| Budget Allocation w/ V0Backbone=Qwen3-4B-Instruct-2507, budget_allocation_mode=real-time difficulty estimation2026.02 | 0.5634 | — | |
| Dr.GRPO-7B [R1-Distill]Backbone=DeepSeek-R1-Distill-Qwen-7B, Optimization Algorithm=Dr.GRPO2026.01 | 0.557 | — | |
| Budget Allocation w/o V0Backbone=Qwen3-4B-Instruct-2507, budget_allocation_mode=lagged heuristics2026.02 | 0.5497 | — | |
| GRPOBackbone=Qwen3-4B-Instruct-25072026.02 | 0.5453 | — | |
| R1-Distill-Qwen-7B-R-TAP2026.03 | 0.538 | — | |
| OpenReasoner-Zero-7B@8kBackbone=Qwen2.5-Math-7B, Optimization Algorithm=OpenReasoner-Zero, Sampling Budget=8k2026.01 | 0.479 | — | |
| OpenReasoner-Zero-7B @ 8kTraining iterations=8k2026.03 | 0.479 | — | |
| R1-Distill-Qwen-7B @ 8kTraining iterations=8k2026.03 | 0.477 | — | |
| Oat-Zero-7B-R-TAP2026.03 | 0.468 | — | |
| WISTBackbone=Qwen3-14B-Base2026.03 | 0.468 | — | |
| PMPO-7B (Ours)Backbone=Qwen2.5-Math-7B, Optimization Algorithm=PMPO2026.01 | 0.467 | — | |
| SPICEBackbone=Qwen3-14B-Base2026.03 | 0.467 | — | |
| SPICEBackbone=Qwen3-8B-Base2026.03 | 0.453 | — | |
| R-ZeroBackbone=Qwen3-8B-Base2026.03 | 0.452 | — | |
| R-ZeroBackbone=Qwen3-14B-Base2026.03 | 0.452 | — | |
| WISTBackbone=Qwen3-8B-Base2026.03 | 0.441 | — | |
| OpenReasoner-Zero-7B@3kBackbone=Qwen2.5-Math-7B, Optimization Algorithm=OpenReasoner-Zero, Sampling Budget=3k2026.01 | 0.44 | — | |
| OpenReasoner-Zero-7B @ 3kTraining iterations=3k2026.03 | 0.44 | — | |
| GMPO-7BBackbone=Qwen2.5-Math-7B, Optimization Algorithm=GMPO2026.01 | 0.436 | — | |
| Base ModelBackbone=Qwen3-14B-Base2026.03 | 0.431 | — | |
| Oat-Zero-1.5B-R-TAP2026.03 | 0.428 | — | |
| Qwen2.5-Math-7B-Instruct2026.03 | 0.427 | — | |
| GPG-7BBackbone=Qwen2.5-Math-7B, Optimization Algorithm=GPG2026.01 | 0.424 | — | |
| R1-Distill-Qwen-1.5B-R-TAP @ 8kTraining iterations=8k2026.03 | 0.424 | — | |
| Dr.GRPO-7BBackbone=Qwen2.5-Math-7B, Optimization Algorithm=Dr.GRPO2026.01 | 0.41 | — | |
| Oat-Zero-7B2026.03 | 0.41 | — | |
| SPICEBackbone=Qwen3-4B-Base2026.03 | 0.41 | — | |
| PRIME-Zero-7BBackbone=Qwen2.5-Math-7B, Optimization Algorithm=PRIME-Zero2026.01 | 0.409 | — | |
| Eurus-7BBackbone=Mistral-7B, Optimization Algorithm=Eurus2026.01 | 0.409 | — | |
| PRIME-Zero-7B2026.03 | 0.409 | — | |
| Base ModelBackbone=Qwen3-8B-Base2026.03 | 0.405 | — | |
| SimpleRL-Zero-7BBackbone=Qwen2.5-Math-7B, Optimization Algorithm=SimpleRL-Zero2026.01 | 0.403 | — | |
| SimpleRL-Zero-7B2026.03 | 0.403 | — | |
| Qwen2.5-Math-1.5B-InstructBackbone=Qwen2.5-Math-1.5B, Training Strategy=Instruct2026.01 | 0.402 | — | |
| Qwen2.5-Math-1.5B-Instruct2026.03 | 0.402 | — | |
| WISTBackbone=Qwen3-4B-Base2026.03 | 0.4 | — | |
| R-ZeroBackbone=Qwen3-4B-Base2026.03 | 0.395 | — | |
| PMPO-1.5B (Ours)Backbone=Qwen2.5-Math-1.5B, Optimization Algorithm=PMPO2026.01 | 0.393 | — | |
| GMPO-1.5BBackbone=Qwen2.5-Math-1.5B, Optimization Algorithm=GMPO2026.01 | 0.387 | — | |
| Dr.GRPO-1.5BBackbone=Qwen2.5-Math-1.5B, Optimization Algorithm=Dr.GRPO2026.01 | 0.376 | — | |
| Oat-Zero-1.5B2026.03 | 0.376 | — | |
| R1-Distill-Qwen-1.5B @ 8kTraining iterations=8k2026.03 | 0.358 | — | |
| Qwen2.5-Math-7B2026.03 | 0.347 | — | |
| Base ModelBackbone=Qwen3-4B-Base2026.03 | 0.302 | — | |
| Qwen2.5-Math-1.5BBackbone=Qwen2.5-Math-1.5B2026.01 | 0.284 | — | |
| Qwen2.5-Math-1.5B2026.03 | 0.284 | — | |
| GRPOModel=Qwen3-8B-Base2025.11 | 0.26 | — | |
| OBLR-POModel=Qwen3-8B-Base2025.11 | 0.26 | — | |
| R-ZeroBackbone=OctoThinker-8B-Hybrid-Base2026.03 | 0.259 | — | |
| WISTBackbone=OctoThinker-8B-Hybrid-Base2026.03 | 0.255 | — | |
| ReMaxModel=Qwen3-8B-Base2025.11 | 0.253 | — | |
| SPICEBackbone=OctoThinker-8B-Hybrid-Base2026.03 | 0.249 | — | |
| RLOOModel=Qwen3-8B-Base2025.11 | 0.247 | — | |
| OBLR-POModel=Qwen3-4B-Base2025.11 | 0.241 | — | |
| PPOModel=Qwen3-8B-Base2025.11 | 0.238 | — | |
| GRPOModel=Qwen3-4B-Base2025.11 | 0.235 | — | |
| R1-Distill-Qwen-7B @ 3kTraining iterations=3k2026.03 | 0.231 | — | |
| RLOOModel=Qwen3-4B-Base2025.11 | 0.226 | — | |
| ReMaxModel=Qwen3-4B-Base2025.11 | 0.224 | — | |
| PPOModel=Qwen3-4B-Base2025.11 | 0.22 | — | |
| R1-Distill-Qwen-1.5B @ 3kTraining iterations=3k2026.03 | 0.173 | — | |
| SPICEBackbone=OctoThinker-3B-Hybrid-Base2026.03 | 0.173 | — | |
| R-ZeroBackbone=OctoThinker-3B-Hybrid-Base2026.03 | 0.172 | — | |
| Base ModelBackbone=OctoThinker-8B-Hybrid-Base2026.03 | 0.17 | — | |
| Qwen2.5-Math-7BBackbone=Qwen2.5-Math-7B2026.01 | 0.166 | — | |
| WISTBackbone=OctoThinker-3B-Hybrid-Base2026.03 | 0.151 | — | |
| Base ModelBackbone=OctoThinker-3B-Hybrid-Base2026.03 | 0.11 | — | |
| BaseModel=DeepSeek-R1-Distill-Qwen-1.5B, beta=N/A2025.09 | — | 0.5096 | |
| BaseModel=Llama-3.2-1B-Instruct, beta=N/A2025.09 | — | 0.0563 | |
| GRPOModel=DeepSeek-R1-Distill-Qwen-1.5B, beta=02025.09 | — | 0.4444 | |
| GRPOModel=DeepSeek-R1-Distill-Qwen-1.5B, beta=0.042025.09 | — | 0.52 | |
| GRPOModel=Llama-3.2-1B-Instruct, beta=02025.09 | — | 0.0607 | |
| GRPOModel=Llama-3.2-1B-Instruct, beta=0.042025.09 | — | 0.0533 | |
| λ-GRPOModel=DeepSeek-R1-Distill-Qwen-1.5B, beta=02025.09 | — | 0.5348 | |
| λ-GRPOModel=DeepSeek-R1-Distill-Qwen-1.5B, beta=0.042025.09 | — | 0.5378 | |
| λ-GRPOModel=Llama-3.2-1B-Instruct, beta=02025.09 | — | 0.0622 | |
| λ-GRPOModel=Llama-3.2-1B-Instruct, beta=0.042025.09 | — | 0.0489 |