Mathematical Reasoning on GSM8K v1 (test)
95.1AccuracyPass@k
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Pass@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 95.1 | — | — | |
| Large greedyModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 95.1 | — | — | |
| Large greedyModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 95.1 | — | — | |
| Large greedyModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 95.1 | — | — | |
| NEMOTRON-NANO-12B-V2 (RLP + Post)Pre-training objective=RLP, Post-training pipeline=SFT + RLVR, Evaluation protocol=Pass@1 (average of 8 runs)2025.09 | 94.9 | — | — | |
| Large greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 94.8 | — | — | |
| Large greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 94.8 | — | — | |
| Large greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 94.8 | — | — | |
| NEMOTRON-NANO-12B-V2 (Base + Post)Pre-training objective=Next-token prediction, Post-training pipeline=SFT + RLVR, Evaluation protocol=Pass@1 (average of 8 runs)2025.09 | 94.22 | — | — | |
| Vanilla SPDraft Model=Llama-3.1-8B-Instruct, Target Model=Llama-3.1-70B-Instruct2026.01 | 93 | 24.19 | — | |
| AutoJudgeDraft Model=Llama-3.1-8B-Instruct, Target Model=Llama-3.1-70B-Instruct, Threshold=0.042026.01 | 92.77 | 27.34 | 1.13 | |
| Pass@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 92.7 | — | — | |
| KLDraft Model=Llama-3.1-8B-Instruct, Target Model=Llama-3.1-70B-Instruct, Threshold=0.202026.01 | 92.69 | 27.21 | 1.12 | |
| AutoJudgeDraft Model=Llama-3.1-8B-Instruct, Target Model=Llama-3.1-70B-Instruct, Threshold=0.072026.01 | 92.54 | 29.55 | 1.22 | |
| CGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=32, Chunk Length (L)=202026.06 | 92.5 | — | — | |
| PRM guided searchModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 92.3 | — | — | |
| AutoJudgeDraft Model=Llama-3.1-8B-Instruct, Target Model=Llama-3.1-70B-Instruct, Threshold=0.092026.01 | 92.09 | 30.98 | 1.28 | |
| KLDraft Model=Llama-3.1-8B-Instruct, Target Model=Llama-3.1-70B-Instruct, Threshold=0.402026.01 | 91.94 | 30.1 | 1.24 | |
| KLDraft Model=Llama-3.1-8B-Instruct, Target Model=Llama-3.1-70B-Instruct, Threshold=0.502026.01 | 91.88 | 31.49 | 1.3 | |
| KLDraft Model=Llama-3.1-8B-Instruct, Target Model=Llama-3.1-70B-Instruct, Threshold=0.902026.01 | 91.28 | 33.21 | 1.37 | |
| AutoJudgeDraft Model=Llama-3.1-8B-Instruct, Target Model=Llama-3.1-70B-Instruct, Threshold=0.142026.01 | 91.08 | 32.57 | 1.35 | |
| LGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=32, Chunk Length (L)=202026.06 | 90.4 | — | — | |
| KLDraft Model=Llama-3.1-8B-Instruct, Target Model=Llama-3.1-70B-Instruct, Threshold=1.402026.01 | 90.27 | 35.1 | 1.45 | |
| AutoJudgeDraft Model=Llama-3.1-8B-Instruct, Target Model=Llama-3.1-70B-Instruct, Threshold=0.222026.01 | 89.96 | 35.07 | 1.45 | |
| CGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=16, Chunk Length (L)=202026.06 | 89.3 | — | — | |
| PRM guided searchModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 89.2 | — | — | |
| Pass@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 88.6 | — | — | |
| LGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=16, Chunk Length (L)=202026.06 | 87.6 | — | — | |
| PROMPTCOT-Qwen-1.5BCoT setting=short-CoT, Base model=Qwen2.5-Math-1.5B2025.03 | 87.1 | — | — | |
| Pass@kModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 86.8 | — | — | |
| PROMPTCOT-DS-1.5BCoT setting=long-CoT, Base model=DeepSeek-R1-Distill-Qwen-1.5B2025.03 | 86.1 | — | — | |
| PRM guided searchModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 86 | — | — | |
| NEMOTRON-NANO-12B-V2 (RLP)Pre-training objective=RLP, Post-training pipeline=None, Evaluation protocol=Pass@1 (average of 8 runs)2025.09 | 85.98 | — | — | |
| Qwen2-7BInstruction-tuned=true2024.07 | 85.7 | — | — | |
| CGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=8, Chunk Length (L)=202026.06 | 85.7 | — | — | |
| LGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=8, Chunk Length (L)=202026.06 | 85.4 | — | — | |
| NuminaMath-1.5BCoT setting=short-CoT, Base model=Qwen2.5-Math-1.5B2025.03 | 85.1 | — | — | |
| DeepSeek-R1-Distill-Qwen-1.5BCoT setting=long-CoT, reproduced_using_our_prompt=true2025.03 | 85.1 | — | — | |
| Openmathinstruct-1.5BCoT setting=short-CoT, Base model=Qwen2.5-Math-1.5B2025.03 | 84.9 | — | — | |
| Yi-1.5-9BInstruction-tuned=true2024.07 | 84.8 | — | — | |
| Qwen2.5-Math-1.5B-InstructCoT setting=short-CoT, Base model=N/A2025.03 | 84.8 | — | — | |
| Evol-Instruct-1.5BCoT setting=short-CoT, Base model=Qwen2.5-Math-1.5B2025.03 | 84.6 | — | — | |
| TABOMSFT Training Domain=Math SFT (MixChain-Z-PRM12K)2026.05 | 84.31 | — | — | |
| CGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=32, Chunk Length (L)=202026.06 | 83.9 | — | — | |
| KPDDS-1.5BCoT setting=short-CoT, Base model=Qwen2.5-Math-1.5B2025.03 | 83.4 | — | — | |
| KLDraft Model=Llama-3.2-1B-Instruct, Target Model=Llama-3.1-8B-Instruct, Threshold=0.102026.01 | 82.77 | 39.58 | 1.03 | |
| AutoJudgeDraft Model=Llama-3.2-1B-Instruct, Target Model=Llama-3.1-8B-Instruct, Threshold=0.012026.01 | 82.5 | 39.11 | 1.02 | |
| Vanilla SPDraft Model=Llama-3.2-1B-Instruct, Target Model=Llama-3.1-8B-Instruct2026.01 | 82.5 | 38.4 | — | |
| SFT-SDSFT Training Domain=Math SFT (MixChain-Z-PRM12K)2026.05 | 81.95 | — | — | |
| SFT-SDSFT Training Domain=Code SFT (Ling-Coder-SFT)2026.05 | 81.81 | — | — | |
| TABOMSFT Training Domain=Code SFT (Ling-Coder-SFT)2026.05 | 81.73 | — | — | |
| No-SFTSFT Training Domain=None (Base Model)2026.05 | 81.41 | — | — | |
| LGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=32, Chunk Length (L)=202026.06 | 81.2 | — | — | |
| KLDraft Model=Llama-3.2-1B-Instruct, Target Model=Llama-3.1-8B-Instruct, Threshold=0.402026.01 | 81.07 | 48.35 | 1.26 | |
| Borda count (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 81 | — | — | |
| KLDraft Model=Llama-3.2-1B-Instruct, Target Model=Llama-3.1-8B-Instruct, Threshold=0.302026.01 | 80.96 | 45.35 | 1.18 | |
| Best-of-NModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 80.7 | — | — | |
| Best-of-NModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 80.5 | — | — | |
| AutoJudgeDraft Model=Llama-3.2-1B-Instruct, Target Model=Llama-3.1-8B-Instruct, Threshold=0.072026.01 | 80.29 | 45.06 | 1.17 | |
| SFT-GTSFT Training Domain=Math SFT (MixChain-Z-PRM12K)2026.05 | 80.12 | — | — | |
| Borda count (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 80.1 | — | — | |
| Majority@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 79.7 | — | — | |
| Llama-3-8BInstruction-tuned=true2024.07 | 79.6 | — | — | |
| GLM-4-9BInstruction-tuned=true2024.07 | 79.6 | — | — | |
| Best-of-NModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 79.6 | — | — | |
| KLDraft Model=Llama-3.2-1B-Instruct, Target Model=Llama-3.1-8B-Instruct, Threshold=0.502026.01 | 79.46 | 50.96 | 1.33 | |
| AutoJudgeDraft Model=Llama-3.2-1B-Instruct, Target Model=Llama-3.1-8B-Instruct, Threshold=0.092026.01 | 79.39 | 47.12 | 1.23 | |
| CGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=16, Chunk Length (L)=202026.06 | 79.3 | — | — | |
| Pass@kModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 79.1 | — | — | |
| Borda count (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 78.7 | — | — | |
| AutoJudgeDraft Model=Llama-3.2-1B-Instruct, Target Model=Llama-3.1-8B-Instruct, Threshold=0.142026.01 | 78.4 | 48.87 | 1.27 | |
| Borda count (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 78 | — | — | |
| Majority@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 77.7 | — | — | |
| LGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=16, Chunk Length (L)=202026.06 | 77.3 | — | — | |
| PRM guided searchModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 77.3 | — | — | |
| KLDraft Model=Llama-3.2-1B-Instruct, Target Model=Llama-3.1-8B-Instruct, Threshold=0.902026.01 | 77.13 | 61.45 | 1.6 | |
| AutoJudgeDraft Model=Llama-3.2-1B-Instruct, Target Model=Llama-3.1-8B-Instruct, Threshold=0.222026.01 | 75.59 | 53.7 | 1.4 | |
| Borda count (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 75.2 | — | — | |
| Borda count (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 75 | — | — | |
| Self-Certainty (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 74.8 | — | — | |
| Majority@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 74 | — | — | |
| Self-Certainty (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 73.4 | — | — | |
| NEMOTRON-NANO-12B-V2 (Base)Pre-training objective=Next-token prediction, Post-training pipeline=None, Evaluation protocol=Pass@1 (average of 8 runs)2025.09 | 72.31 | — | — | |
| PRM guided searchModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 71.8 | — | — | |
| CGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=8, Chunk Length (L)=202026.06 | 71.3 | — | — | |
| Pass@kModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 70.5 | — | — | |
| LGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=8, Chunk Length (L)=202026.06 | 69.8 | — | — | |
| Self-Certainty (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 69.7 | — | — | |
| Self-Certainty (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 67.9 | — | — | |
| Self-Certainty (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 67.5 | — | — | |
| PRM guided searchModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 65.4 | — | — | |
| Self-Certainty (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 65.1 | — | — | |
| Self-Certainty (large)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 62.7 | — | — | |
| Qwen1.5-7BInstruction-tuned=true2024.07 | 60.3 | — | — | |
| Best-of-NModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 60.2 | — | — | |
| Self-Certainty (large)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 59.2 | — | — | |
| Borda count (large)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 59.1 | — | — | |
| Best-of-NModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 58.5 | — | — | |
| Borda count (small)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 57.2 | — | — | |
| Borda count (large)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 57 | — | — |