Mathematical Reasoning on Minerva (Acc, Cost)
89.16Accuracy (Acc)COSE
Evaluation results
| Method | Links | ||
|---|---|---|---|
| COSEBackbone=Llama-3.2-3B-Instruct2026.05 | 89.16 | — | |
| COSEBackbone=Qwen3-4B2026.05 | 88.38 | — | |
| MAEBackbone=Qwen3-4B2026.05 | 81.8 | — | |
| R-ZeroBackbone=Qwen3-4B2026.05 | 81.2 | — | |
| AZRBackbone=Qwen3-4B2026.05 | 81.1 | — | |
| BaseBackbone=Qwen3-4B2026.05 | 78.7 | — | |
| COSEBackbone=Qwen3-0.6B2026.05 | 78.18 | — | |
| MAEBackbone=Llama-3.2-3B-Instruct2026.05 | 70.2 | — | |
| R-ZeroBackbone=Llama-3.2-3B-Instruct2026.05 | 69.1 | — | |
| AZRBackbone=Llama-3.2-3B-Instruct2026.05 | 68.4 | — | |
| BaseBackbone=Llama-3.2-3B-Instruct2026.05 | 64.1 | — | |
| MAEBackbone=Qwen3-0.6B2026.05 | 55.4 | — | |
| R-ZeroBackbone=Qwen3-0.6B2026.05 | 54.8 | — | |
| AZRBackbone=Qwen3-0.6B2026.05 | 53.6 | — | |
| DYPOBase Model=Qwen3-4B-Base2026.04 | 50.4 | — | |
| BaseBackbone=Qwen3-0.6B2026.05 | 48.8 | — | |
| COSEBackbone=Qwen2.5-3B-Instruct2026.05 | 47.02 | — | |
| RLBase Model=Qwen3-4B-Base2026.04 | 46.3 | — | |
| ICPOBackbone=Qwen2.5-Math-7B2025.10 | 45.6 | — | |
| OGERBackbone=Qwen2.5-Math-7B2026.04 | 45.22 | — | |
| SFT -> RLBase Model=Qwen3-4B-Base2026.04 | 44.9 | — | |
| ICPO†Backbone=Qwen2.5-Math-7B2025.10 | 44.9 | — | |
| SFT+RL_GRPOBackbone=Qwen2.5-Math-7B2025.10 | 44.1 | — | |
| Re-ScheduleBackbone=Qwen3-4B-Base2025.10 | 43.5 | — | |
| SFTBase Model=Qwen3-4B-Base2026.04 | 43 | — | |
| MAEBackbone=Qwen2.5-3B-Instruct2026.05 | 42.6 | — | |
| LuffyBackbone=Qwen2.5-Math-7B2026.04 | 42.28 | — | |
| Task ArithmeticModel Scale=7B, Decoding Strategy=Greedy2026.03 | 41.9 | — | |
| FIM-TIESModel Scale=7B, Decoding Strategy=Greedy2026.03 | 41.9 | — | |
| ACCBackbone=Qwen3-4B-Base2025.10 | 41.6 | — | |
| Sens-MergingModel Scale=7B, Decoding Strategy=Greedy2026.03 | 41.5 | — | |
| AZRBackbone=Qwen2.5-3B-Instruct2026.05 | 41.22 | — | |
| FIM-TAModel Scale=7B, Decoding Strategy=Greedy2026.03 | 41.2 | — | |
| R-ZeroBackbone=Qwen2.5-3B-Instruct2026.05 | 40.9 | — | |
| Prefix-RFT*Backbone=Qwen2.5-Math-7B, Reported from original paper=true2025.10 | 40.3 | — | |
| GRPOBackbone=Qwen2.5-Math-7B2026.04 | 39.71 | — | |
| GRPOBackbone=Qwen3-4B-Base2025.10 | 39.4 | — | |
| PRIME-Zero*Backbone=Qwen2.5-Math-7B, Reported from original paper=true2025.10 | 39 | — | |
| GRPObase_model=Qwen2.5-Math-7B2026.05 | 38.75 | — | |
| ACM-TAModel Scale=7B, Decoding Strategy=Greedy2026.03 | 38.6 | — | |
| ACM-TIESModel Scale=7B, Decoding Strategy=Greedy2026.03 | 38.6 | — | |
| TIES-MergingModel Scale=7B, Decoding Strategy=Greedy2026.03 | 38.2 | — | |
| CoDistill-GRPObase_model=Qwen2.5-Math-7B, alpha=1, T=0, CT=true2026.05 | 38.16 | — | |
| SFTBackbone=Qwen2.5-Math-7B2025.10 | 37.9 | — | |
| LUFFY*Backbone=Qwen2.5-Math-7B, Reported from original paper=true2025.10 | 37.5 | — | |
| CoDistill-GRPObase_model=Qwen2.5-Math-7B, alpha=1, T=50, CT=true2026.05 | 37.28 | — | |
| RL_GRPOBackbone=Qwen2.5-Math-7B2025.10 | 37.1 | — | |
| ReLIFT*Backbone=Qwen2.5-Math-7B, Reported from original paper=true2025.10 | 37.1 | — | |
| DeepSeek-R1-7BModel Scale=7B, Decoding Strategy=Greedy2026.03 | 36.4 | — | |
| CoDistill-GRPObase_model=Qwen2.5-Math-7B, alpha=2, T=0, CT=true2026.05 | 36 | — | |
| CoDistill-GRPObase_model=Qwen2.5-Math-7B, alpha=1, T=50, CT=false2026.05 | 35.51 | — | |
| CoDistill-GRPObase_model=Qwen2.5-Math-7B, alpha=2, T=0, CT=false2026.05 | 35.15 | — | |
| KAEBackbone=Qwen2.5-Math-7B, Post-training Dataset=MATH (levels 3-5, 5k+), Training Steps=200, Group Size=G = 8, Temperature=0.62026.04 | 34.93 | — | |
| Oat-Zero*Backbone=Qwen2.5-Math-7B, Reported from original paper=true2025.10 | 34.6 | — | |
| CoDistill-GRPObase_model=Qwen2.5-Math-7B, alpha=1, T=0, CT=false2026.05 | 34.56 | — | |
| BaseBackbone=Qwen2.5-3B-Instruct2026.05 | 34.52 | — | |
| OpenReasoner-Zero*Backbone=Qwen2.5-Math-7B, Reported from original paper=true2025.10 | 33.1 | — | |
| Dr. GRPOBackbone=Qwen2.5-Math-7B, Post-training Dataset=MATH (levels 3-5, 5k+), Training Steps=200, Group Size=G = 8, Temperature=0.62026.04 | 33.09 | — | |
| GRPO-SGBase Model=Qwen2.5-7B, Training Recipe=DSR (DeepScaleR)2025.10 | 32.35 | — | |
| CoDistill-GRPOBase Model=Qwen2.5-Math-1.5B, alpha=2, T=02026.05 | 31.99 | — | |
| GRPOBackbone=Qwen2.5-Math-7B, Post-training Dataset=MATH (levels 3-5, 5k+), Training Steps=200, Group Size=G = 8, Temperature=0.62026.04 | 31.62 | — | |
| CoTModel=Gemma32026.03 | 31.4 | 3,801 | |
| GRPO w/ LoRA#Steps=400, Backbone=Qwen2.5-14B-Instruct2026.04 | 31.3 | — | |
| GPGBackbone=Qwen2.5-Math-7B, Post-training Dataset=MATH (levels 3-5, 5k+), Training Steps=200, Group Size=G = 8, Temperature=0.62026.04 | 31.25 | — | |
| GRPO-SGBase Model=Qwen2.5-7B, Training Recipe=ORZ (Open Reasoner-Zero)2025.10 | 30.51 | — | |
| NExt#Steps=250, Backbone=Qwen2.5-14B-Instruct2026.04 | 30.5 | — | |
| DoTS (SFT + DPO)Backbone=Qwen2.5-Math-1.5B, Training Configuration Source=DFT (Wu et al., 2025b)2026.05 | 30.5 | — | |
| LUFFYBackbone=Qwen2.5-Math-1.5B, Training Configuration Source=LUFFY (Yan et al., 2025)2026.05 | 30.5 | — | |
| ARBase Model=Qwen2.5-7B, Training Recipe=ORZ (Open Reasoner-Zero)2025.10 | 30.22 | — | |
| RL-Extra#Steps=250, Backbone=Qwen2.5-14B-Instruct2026.04 | 30.2 | — | |
| LoptiBase Model=Qwen2.5-7B, Training Recipe=ORZ (Open Reasoner-Zero)2025.10 | 29.81 | — | |
| GRPO w/ LoRA#Steps=250, Backbone=Qwen2.5-14B-Instruct2026.04 | 29.7 | — | |
| LoptiBase Model=Qwen2.5-7B, Training Recipe=DSR (DeepScaleR)2025.10 | 29.56 | — | |
| OGERBackbone=Qwen2.5-Math-1.5B2026.04 | 29.41 | — | |
| GRPO w/ FP#Steps=250, Backbone=Qwen2.5-14B-Instruct2026.04 | 29.4 | — | |
| 80/20Base Model=Qwen2.5-7B, Training Recipe=ORZ (Open Reasoner-Zero)2025.10 | 29.31 | — | |
| CoDistill-GRPOBase Model=Qwen2.5-Math-1.5B, alpha=1, T=02026.05 | 29.12 | — | |
| ARBase Model=Qwen2.5-7B, Training Recipe=DSR (DeepScaleR)2025.10 | 29.11 | — | |
| GRPO w/ FP#Steps=400, Backbone=Qwen2.5-14B-Instruct2026.04 | 29 | — | |
| DoTS (SFT + On-Policy RL)Backbone=Qwen2.5-Math-1.5B, Training Configuration Source=LUFFY (Yan et al., 2025)2026.05 | 29 | — | |
| Qwen2.5-Math-1.5B-InstructBackbone=Qwen2.5-Math-1.5B2026.05 | 28.7 | — | |
| SFTBackbone=Qwen2.5-Math-1.5B, Training Configuration Source=LUFFY (Yan et al., 2025)2026.05 | 28.7 | — | |
| GRPOBase Model=Qwen2.5-7B, Training Recipe=DSR (DeepScaleR)2025.10 | 28.68 | — | |
| AlphaRL#Steps=250, Backbone=Qwen2.5-14B-Instruct2026.04 | 28.6 | — | |
| On-Policy RLBackbone=Qwen2.5-Math-1.5B, Training Configuration Source=LUFFY (Yan et al., 2025)2026.05 | 28.3 | — | |
| CoDistill-GRPOBase Model=Qwen2.5-Math-1.5B, alpha=1, T=502026.05 | 28.3 | — | |
| STEERBackbone=Qwen2.5-Math-1.5B, Sampling Strategy=avg@12025.10 | 28.2 | — | |
| GRPOBase Model=Qwen2.5-7B, Training Recipe=ORZ (Open Reasoner-Zero)2025.10 | 27.94 | — | |
| 80/20Base Model=Qwen2.5-7B, Training Recipe=DSR (DeepScaleR)2025.10 | 27.92 | — | |
| Qwen3-4B-BaseBase Model=Qwen3-4B-Base2026.04 | 27.9 | — | |
| LuffyBackbone=Qwen2.5-Math-1.5B2026.04 | 27.21 | — | |
| KDRL w/ 7BBase Model=Qwen2.5-Math-1.5B, Teacher Model=7B2026.05 | 27.13 | — | |
| CoTModel=Qwen3-8B2026.03 | 26.8 | 3,320 | |
| Entro. Adv.Backbone=Qwen2.5-Math-1.5B, Sampling Strategy=avg@12025.10 | 26.8 | — | |
| GRPOBackbone=Qwen2.5-Math-1.5B2026.04 | 26.47 | — | |
| Clip-CovBackbone=Qwen2.5-Math-1.5B, Sampling Strategy=avg@12025.10 | 26.4 | — | |
| Backbone Model#Steps=-, Backbone=Qwen2.5-14B-Instruct2026.04 | 26.1 | — | |
| GRPOBackbone=Qwen2.5-Math-1.5B, Sampling Strategy=avg@12025.10 | 26.1 | — | |
| OPOBackbone=Qwen2.5-Math-1.5B, Sampling Strategy=avg@12025.10 | 26.1 | — | |
| CoTModel=Llama3.12026.03 | 25.95 | 3,416 |