Mathematical Reasoning on Minerva (test)
45AccSolver + Verifier (AstraFlow)
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Solver + Verifier (AstraFlow)Backbone=Qwen3-8B, Framework=AstraFlow, Time/iter (s)=77.652026.05 | 45 | — | — | — | — | — | |
| AgentPSOCategory=Multi-Agent2026.05 | 42.65 | — | — | — | — | — | |
| Solver + Verifier (verl)Backbone=Qwen3-8B, Framework=verl, Time/iter (s)=212.642026.05 | 40.9 | — | — | — | — | — | |
| SolverBackbone=Qwen3-8B2026.05 | 39.2 | — | — | — | — | — | |
| Chain-of-ThoughtCategory=Vanilla Single-Agent2026.05 | 38.6 | — | — | — | — | — | |
| Step-Back PromptingCategory=Vanilla Single-Agent2026.05 | 38.6 | — | — | — | — | — | |
| ReflectionCategory=Advanced Single-Agent2026.05 | 38.24 | — | — | — | — | — | |
| DMADCategory=Multi-Agent2026.05 | 35.29 | — | — | — | — | — | |
| Self-RefineCategory=Advanced Single-Agent2026.05 | 34.56 | — | — | — | — | — | |
| MADCategory=Multi-Agent2026.05 | 32.72 | — | — | — | — | — | |
| ToTCategory=Advanced Single-Agent2026.05 | 31.99 | — | — | — | — | — | |
| ThinkPrune+CLOREBackbone=DeepSeek-R1-Distill-Qwen-7B2026.05 | 29.4 | 1,408 | — | — | — | 2.29 | |
| GRPOBackbone=DeepSeek-R1-Distill-Qwen-7B2026.05 | 29 | 2,066 | — | — | — | 1.21 | |
| DAPOBackbone=DeepSeek-R1-Distill-Qwen-7B2026.05 | 29 | 1,528 | — | — | — | 1.99 | |
| ThinkPruneBackbone=Qwen2.5-Math-7B2026.05 | 27.6 | 667 | — | — | — | 5.39 | |
| MoACategory=Multi-Agent2026.05 | 27.57 | — | — | — | — | — | |
| Self-ConsistencyCategory=Advanced Single-Agent2026.05 | 27.21 | — | — | — | — | — | |
| GRPO+CLOREBackbone=Qwen2.5-Math-7B2026.05 | 27.2 | 542 | — | — | — | 6.76 | |
| ThinkPrune+CLOREBackbone=Qwen2.5-Math-7B2026.05 | 27.2 | 569 | — | — | — | 6.39 | |
| GRPO+CLOREBackbone=DeepSeek-R1-Distill-Qwen-7B2026.05 | 26.9 | 1,629 | — | — | — | 1.38 | |
| ThinkPruneBackbone=DeepSeek-R1-Distill-Qwen-7B2026.05 | 26.8 | 1,791 | — | — | — | 1.35 | |
| DAPO+CLOREBackbone=DeepSeek-R1-Distill-Qwen-7B2026.05 | 26.5 | 1,255 | — | — | — | 2.32 | |
| Training Efficient+CLOREBackbone=DeepSeek-R1-Distill-Qwen-7B2026.05 | 26.5 | 1,657 | — | — | — | 1.52 | |
| Training EfficientBackbone=DeepSeek-R1-Distill-Qwen-7B2026.05 | 25.7 | 1,862 | — | — | — | 1.17 | |
| GRPOBackbone=Qwen2.5-Math-7B2026.05 | 25.7 | 1,199 | — | — | — | 2.31 | |
| DAPO+CLOREBackbone=Qwen2.5-Math-7B2026.05 | 25.4 | 513 | — | — | — | 6.65 | |
| FGOModel=ZR1-1.5B2026.02 | 24.6 | 412 | 6 | — | — | — | |
| Base model (w/o training)Backbone=DeepSeek-R1-Distill-Qwen-7B2026.05 | 24.6 | 2,958 | — | — | — | — | |
| GRPOModel=ZR1-1.5B2026.02 | 23.2 | 1,209 | 1.9 | — | — | — | |
| FGOModel=Qwen2.5-Math-1.5B-Instruct2026.02 | 23.2 | 374 | 6.2 | — | — | — | |
| GRPOModel=Qwen2.5-Math-1.5B-Instruct2026.02 | 22.1 | 645 | 3.4 | — | — | — | |
| Training Efficient+CLOREBackbone=Qwen2.5-Math-7B2026.05 | 22.1 | 199 | — | — | — | 16.16 | |
| Training EfficientBackbone=Qwen2.5-Math-7B2026.05 | 21.7 | 314 | — | — | — | 9.68 | |
| GRPOModel=DeepSeek-R1-Distill-Qwen-1.5B2026.02 | 20.6 | 1,132 | 1.8 | — | — | — | |
| VanillaModel=Qwen2.5-Math-1.5B-Instruct2026.02 | 20.2 | 652 | 3.1 | — | — | — | |
| FGOModel=Qwen2.5-Math-1.5B2026.02 | 18 | 673 | 2.7 | — | — | — | |
| FGOModel=DeepSeek-R1-Distill-Qwen-1.5B2026.02 | 18 | 202 | 8.9 | — | — | — | |
| VanillaModel=ZR1-1.5B2026.02 | 17.3 | 1,235 | 1.4 | — | — | — | |
| DAPOBackbone=Qwen2.5-Math-7B2026.05 | 15.4 | 555 | — | — | — | 3.29 | |
| VanillaModel=DeepSeek-R1-Distill-Qwen-1.5B2026.02 | 12.1 | 1,404 | 0.9 | — | — | — | |
| GRPOModel=Qwen2.5-Math-1.5B2026.02 | 11.4 | 923 | 1.2 | — | — | — | |
| VanillaModel=Qwen2.5-Math-1.5B2026.02 | 9.2 | 1,357 | 0.7 | — | — | — | |
| Llama3.2-1B-InstructAlgorithm=Llama3.2-1B-Instruct2026.05 | 6.465 | — | — | — | — | — | |
| CoDistill-GRPOAlgorithm=CoDistill-GRPO, Base Model=Llama3.2-1B-Instruct, alpha=1, T=02026.05 | 6.396 | — | — | — | — | — | |
| GRPOAlgorithm=GRPO, Base Model=Llama3.2-1B-Instruct2026.05 | 5.664 | — | — | — | — | — | |
| Base model (w/o training)Backbone=Qwen2.5-Math-7B2026.05 | 5.5 | 850 | — | — | — | — | |
| CHORD-ϕShot Configuration=128-shot, RL on Gold Dataset (DE)=false2026.05 | — | — | — | — | 31.76 | — | |
| DAPOModel Category=RL Post-Trained Models2026.05 | — | — | — | 21.7 | — | — | |
| FEST-DPOShot Configuration=128-shot, RL on Gold Dataset (DE)=false2026.05 | — | — | — | — | 33.18 | — | |
| FEST-GRPOShot Configuration=128-shot, RL on Gold Dataset (DE)=false2026.05 | — | — | — | — | 35.45 | — | |
| GRPOModel Category=RL Post-Trained Models2026.05 | — | — | — | 21 | — | — | |
| GSPOModel Category=RL Post-Trained Models2026.05 | — | — | — | 23.2 | — | — | |
| HölderPOModel Category=RL Post-Trained Models, Schedule=Linear Des: 2 → −22026.05 | — | — | — | 39 | — | — | |
| HPTShot Configuration=128-shot, RL on Gold Dataset (DE)=false2026.05 | — | — | — | — | 33.13 | — | |
| HPT-GShot Configuration=128-shot, RL on Gold Dataset (DE)=true2026.05 | — | — | — | — | 30.51 | — | |
| LUFFYShot Configuration=128-shot, RL on Gold Dataset (DE)=false2026.05 | — | — | — | — | 30.28 | — | |
| Qwen3-4B-BaseModel Category=Base Model2026.05 | — | — | — | 14 | — | — | |
| ReLIFTShot Configuration=128-shot, RL on Gold Dataset (DE)=false2026.05 | — | — | — | — | 34.79 | — | |
| ReLIFT-GShot Configuration=128-shot, RL on Gold Dataset (DE)=true2026.05 | — | — | — | — | 32.44 | — | |
| RLShot Configuration=128-shot, RL on Gold Dataset (DE)=false2026.05 | — | — | — | — | 33.78 | — | |
| RL-GShot Configuration=128-shot, RL on Gold Dataset (DE)=true2026.05 | — | — | — | — | 33.36 | — | |
| SRFTShot Configuration=Full dataset, RL on Gold Dataset (DE)=false2026.05 | — | — | — | — | 29.09 | — |