Mathematical Reasoning on Minerva (Pass@1, Pass@64)
52.9Pass@1GRPOExtraRollouts
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GRPOExtraRolloutsBase Model=8B, Training Method=GRPOExtraRollouts2025.10 | 52.9 | — | |
| ICPO†Base Model=8B, Training Method=ICPO†2025.10 | 51.5 | — | |
| ICPOBase Model=8B, Training Method=ICPO2025.10 | 51.1 | — | |
| GRPOBase Model=8B, Training Method=GRPO2025.10 | 50.7 | — | |
| GRPOExpertDomainBase Model=8B, Training Method=GRPOExpertDomain2025.10 | 50.7 | — | |
| GRPOExpertDomainBase Model=1.7B, Training Method=GRPOExpertDomain2025.10 | 45.6 | — | |
| ICPOBase Model=1.7B, Training Method=ICPO2025.10 | 44.1 | — | |
| GRPOExtraRolloutsBase Model=1.7B, Training Method=GRPOExtraRollouts2025.10 | 43 | — | |
| IPOBackbone=Qwen3-4B-thinking2026.03 | 42.7 | — | |
| ICPO†Base Model=1.7B, Training Method=ICPO†2025.10 | 42.7 | — | |
| SafeKeyBackbone=Qwen3-4B-thinking2026.03 | 42.5 | — | |
| CRAFTBackbone=Qwen3-4B-thinking2026.03 | 42 | — | |
| GRPOBase Model=1.7B, Training Method=GRPO2025.10 | 40.8 | — | |
| BaseBackbone=Qwen3-4B-thinking2026.03 | 40.4 | — | |
| Qwen3Base Model=8B, Training Method=Base2025.10 | 40.4 | — | |
| DeepSearch-1.5Bnumber of samples=32, model scale=1.5B, evaluation cluster=128×H100 96G2025.09 | 40 | — | |
| DPPOBase Model=Qwen3-8B-Base, Algorithm=DPPO2026.05 | 39.9 | — | |
| Nemotron-Research-Reasoning-Qwen-1.5B v2number of samples=32, model scale=1.5B, evaluation cluster=128×H100 96G2025.09 | 39.75 | — | |
| Nemotron-Research-Reasoning-Qwen-1.5B v1number of samples=32, model scale=1.5B, evaluation cluster=128×H100 96G2025.09 | 39.27 | — | |
| NFPOBase Model=Qwen3-8B-Base, Algorithm=NFPO2026.05 | 39 | — | |
| OXAFullModel Backbone=Qwen2.5-7B-Math, RLVR Stage=After RLVR (†)2026.03 | 38.9 | 66.9 | |
| OXAMLEModel Backbone=Qwen2.5-7B-Math, RLVR Stage=After RLVR (†)2026.03 | 38.7 | 64.3 | |
| OXAMLEModel Backbone=Qwen2.5-7B-Math, RLVR Stage=Before RLVR2026.03 | 37.3 | 65.4 | |
| Qwen3Base Model=1.7B, Training Method=Base2025.10 | 37.1 | — | |
| SFTLPModel Backbone=Qwen2.5-7B-Math, RLVR Stage=After RLVR (†)2026.03 | 36.7 | 63.2 | |
| STARBackbone=Qwen3-4B-thinking2026.03 | 36.6 | — | |
| OXAFullModel Backbone=Qwen2.5-7B-Math, RLVR Stage=Before RLVR2026.03 | 36.3 | 64.7 | |
| DeepScaleR-1.5Bnumber of samples=32, model scale=1.5B, evaluation cluster=128×H100 96G2025.09 | 36.19 | — | |
| SafeChainBackbone=Qwen3-4B-thinking2026.03 | 36.1 | — | |
| RealSafeBackbone=Qwen3-4B-thinking2026.03 | 35.8 | — | |
| SFTModel Backbone=Qwen2.5-7B-Math, RLVR Stage=After RLVR (†)2026.03 | 35.1 | 64 | |
| GRPOBase Model=Qwen3-8B-Base, Algorithm=GRPO2026.05 | 35.1 | — | |
| SFTLPModel Backbone=Qwen2.5-7B-Math, RLVR Stage=Before RLVR2026.03 | 34.2 | 62.1 | |
| SFTModel Backbone=Qwen2.5-7B-Math, RLVR Stage=Before RLVR2026.03 | 33.3 | 65.4 | |
| ReasoningShieldBackbone=Qwen3-4B-thinking2026.03 | 33.2 | — | |
| STILL-3-1.5Bnumber of samples=32, model scale=1.5B, evaluation cluster=128×H100 96G2025.09 | 32.77 | — | |
| DeepSeek-R1-Distill-Qwen-1.5Bnumber of samples=32, model scale=1.5B, evaluation cluster=128×H100 96G2025.09 | 32.18 | — | |
| Open-RS1-1.5Bnumber of samples=32, model scale=1.5B, evaluation cluster=128×H100 96G2025.09 | 29.92 | — | |
| Open-RS2-1.5Bnumber of samples=32, model scale=1.5B, evaluation cluster=128×H100 96G2025.09 | 29.74 | — | |
| Qwen2.5-Math-1.5B-Instructnumber of samples=32, model scale=1.5B, evaluation cluster=128×H100 96G2025.09 | 29.32 | — | |
| Open-RS3-1.5Bnumber of samples=32, model scale=1.5B, evaluation cluster=128×H100 96G2025.09 | 29.11 | — | |
| OXAFullModel Backbone=Qwen2.5-1.5B-Math, RLVR Stage=After RLVR (†)2026.03 | 27.6 | 65.1 | |
| OpenSIRBackbone=Llama-3.1-8B-Instruct2025.11 | 27.29 | — | |
| OXAMLEModel Backbone=Qwen2.5-1.5B-Math, RLVR Stage=After RLVR (†)2026.03 | 27.1 | 65.8 | |
| Qwen2.5-Math-1.5B-Oat-Zeronumber of samples=32, model scale=1.5B, evaluation cluster=128×H100 96G2025.09 | 26.84 | — | |
| NFPOBase Model=Qwen3-1.7B-Base, Algorithm=NFPO2026.05 | 26.2 | — | |
| OXAMLEModel Backbone=Qwen2.5-1.5B-Math, RLVR Stage=Before RLVR2026.03 | 25.7 | 63.2 | |
| GRPOgsm8kBackbone=Llama-3.1-8B-Instruct2025.11 | 24.83 | — | |
| DPPOBase Model=Qwen3-1.7B-Base, Algorithm=DPPO2026.05 | 24.6 | — | |
| CRAFTBackbone=DeepSeek-R1-Distill-Llama-8B2026.03 | 24.2 | — | |
| R-ZeroBackbone=Llama-3.1-8B-Instruct2025.11 | 24.11 | — | |
| GRPOmathBackbone=Llama-3.1-8B-Instruct2025.11 | 23.98 | — | |
| IPOBackbone=DeepSeek-R1-Distill-Llama-8B2026.03 | 23.5 | — | |
| SFTModel Backbone=Qwen2.5-1.5B-Math, RLVR Stage=After RLVR (†)2026.03 | 23.3 | 62.5 | |
| Absolute ZeroBackbone=Llama-3.1-8B-Instruct2025.11 | 23.21 | — | |
| SafeKeyBackbone=DeepSeek-R1-Distill-Llama-8B2026.03 | 23.2 | — | |
| ReasoningShieldBackbone=DeepSeek-R1-Distill-Llama-8B2026.03 | 23 | — | |
| OXAFullModel Backbone=Qwen2.5-1.5B-Math, RLVR Stage=Before RLVR2026.03 | 22.8 | 64 | |
| BaseBackbone=Llama-3.1-8B-Instruct2025.11 | 22.75 | — | |
| GRPOBase Model=Qwen3-1.7B-Base, Algorithm=GRPO2026.05 | 22.5 | — | |
| SFTModel Backbone=Qwen2.5-1.5B-Math, RLVR Stage=Before RLVR2026.03 | 22.3 | 61.4 | |
| BaseBackbone=DeepSeek-R1-Distill-Llama-8B2026.03 | 22.1 | — | |
| SFTLPModel Backbone=Qwen2.5-1.5B-Math, RLVR Stage=After RLVR (†)2026.03 | 22 | 58.5 | |
| Qwen3-8B-BaseBase Model=Qwen3-8B-Base, Algorithm=Base2026.05 | 21.9 | — | |
| SFTLPModel Backbone=Qwen2.5-1.5B-Math, RLVR Stage=Before RLVR2026.03 | 20.7 | 61.4 | |
| STARBackbone=DeepSeek-R1-Distill-Llama-8B2026.03 | 20 | — | |
| SafeChainBackbone=DeepSeek-R1-Distill-Llama-8B2026.03 | 19.8 | — | |
| RealSafeBackbone=DeepSeek-R1-Distill-Llama-8B2026.03 | 19.8 | — | |
| OpenSIRBackbone=Llama-3.2-3B-Instruct2025.11 | 18.46 | — | |
| Qwen2.5-Math-1.5Bnumber of samples=32, model scale=1.5B, evaluation cluster=128×H100 96G2025.09 | 18.42 | — | |
| GRPOgsm8kBackbone=Llama-3.2-3B-Instruct2025.11 | 16.27 | — | |
| GRPOmathBackbone=Llama-3.2-3B-Instruct2025.11 | 16.09 | — | |
| R-ZeroBackbone=Llama-3.2-3B-Instruct2025.11 | 15.84 | — | |
| BaseBackbone=Llama-3.2-3B-Instruct2025.11 | 15.21 | — | |
| Absolute ZeroBackbone=Llama-3.2-3B-Instruct2025.11 | 14.78 | — | |
| Qwen3-1.7B-BaseBase Model=Qwen3-1.7B-Base, Algorithm=Base2026.05 | 13.4 | — | |
| BaseModel Backbone=Qwen2.5-7B-Math, RLVR Stage=Before RLVR2026.03 | 13.2 | 64 | |
| BaseModel Backbone=Qwen2.5-1.5B-Math, RLVR Stage=Before RLVR2026.03 | 11.9 | 62.9 |