Mathematical Reasoning on Minerva Math v1 (test)
57.4Accuracy (avg@1)Pass@k
Evaluation Results
| Method | Links | |
|---|---|---|
| Pass@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 57.4 | |
| Pass@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 50.7 | |
| Large greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 50 | |
| Large greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 50 | |
| Large greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 50 | |
| NFTBackbone=Qwen2.5-32B, Fine-tuning Protocol=Supervised fine-tuning2025.05 | 48.9 | |
| DAPOBackbone=Qwen2.5-32B, Fine-tuning Protocol=Reinforcement fine-tuning2025.05 | 47.5 | |
| Large greedyModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 46 | |
| Large greedyModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 46 | |
| Large greedyModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 46 | |
| RFTBackbone=Qwen2.5-32B, Fine-tuning Protocol=Supervised fine-tuning2025.05 | 44.1 | |
| NEMOTRON-NANO-12B-V2 (RLP + Post)Pre-training objective=RLP, Post-training pipeline=SFT + RLVR, Evaluation protocol=Pass@1 (average of 8 runs)2025.09 | 42.78 | |
| NFTBackbone=Qwen2.5-Math-7B, Fine-tuning Protocol=Supervised fine-tuning2025.05 | 40.8 | |
| NEMOTRON-NANO-12B-V2 (Base + Post)Pre-training objective=Next-token prediction, Post-training pipeline=SFT + RLVR, Evaluation protocol=Pass@1 (average of 8 runs)2025.09 | 40.76 | |
| NEMOTRON-NANO-12B-V2 (RLP)Pre-training objective=RLP, Post-training pipeline=None, Evaluation protocol=Pass@1 (average of 8 runs)2025.09 | 39.48 | |
| DAPOBackbone=Qwen2.5-Math-7B, Fine-tuning Protocol=Reinforcement fine-tuning2025.05 | 39.3 | |
| DPOBackbone=Qwen2.5-Math-7B, Fine-tuning Protocol=Preference fine-tuning2025.05 | 39 | |
| CGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=32, Chunk Length (L)=202026.06 | 39 | |
| RFTBackbone=Qwen2.5-Math-7B, Fine-tuning Protocol=Supervised fine-tuning2025.05 | 38.6 | |
| GRPOBackbone=Qwen2.5-Math-7B, Fine-tuning Protocol=Reinforcement fine-tuning2025.05 | 38.2 | |
| Dr. GRPOBackbone=Qwen2.5-Math-7B, Fine-tuning Protocol=Reinforcement fine-tuning2025.05 | 38.2 | |
| Pass@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 38.2 | |
| CGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=16, Chunk Length (L)=202026.06 | 36.8 | |
| LGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=32, Chunk Length (L)=202026.06 | 36 | |
| Pass@kModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 35.3 | |
| LGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=16, Chunk Length (L)=202026.06 | 33.8 | |
| CGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=8, Chunk Length (L)=202026.06 | 32.7 | |
| PRM guided searchModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 32.4 | |
| LGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=8, Chunk Length (L)=202026.06 | 31.6 | |
| Pass@kModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 29.4 | |
| PRM guided searchModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 28.3 | |
| Borda count (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 28.3 | |
| Qwen2.5-32BBackbone=Qwen2.5-32B, Fine-tuning Protocol=Base2025.05 | 27.9 | |
| Borda count (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 26.8 | |
| Self-Certainty (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 26.5 | |
| Majority@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 26.1 | |
| Borda count (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 26.1 | |
| Borda count (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 25.7 | |
| PRM guided searchModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 25 | |
| Majority@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 24.3 | |
| Borda count (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 22.8 | |
| NEMOTRON-NANO-12B-V2 (Base)Pre-training objective=Next-token prediction, Post-training pipeline=None, Evaluation protocol=Pass@1 (average of 8 runs)2025.09 | 22.61 | |
| Self-Certainty (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 22.4 | |
| Borda count (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 22.1 | |
| CGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=32, Chunk Length (L)=202026.06 | 21.7 | |
| Qwen2.5-Math-7BBackbone=Qwen2.5-Math-7B, Fine-tuning Protocol=Base2025.05 | 21.3 | |
| Majority@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 21.3 | |
| Best-of-NModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 21.3 | |
| Pass@kModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 21 | |
| Self-Certainty (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 20.6 | |
| Self-Certainty (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 20.6 | |
| Self-Certainty (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 19.9 | |
| Best-of-NModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 19.5 | |
| Small greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 19.1 | |
| Small greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 19.1 | |
| Small greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 19.1 | |
| Self-Certainty (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 18 | |
| CGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=8, Chunk Length (L)=202026.06 | 16.2 | |
| LGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=16, Chunk Length (L)=202026.06 | 16.2 | |
| Best-of-NModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 15.8 | |
| LGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=32, Chunk Length (L)=202026.06 | 15.4 | |
| CGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=16, Chunk Length (L)=202026.06 | 15.1 | |
| PRM guided searchModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 13.2 | |
| LGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=8, Chunk Length (L)=202026.06 | 12.9 | |
| PRM guided searchModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 12.5 | |
| Self-Certainty (large)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 11.8 | |
| Best-of-NModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 11.4 | |
| PRM guided searchModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 11 | |
| Borda count (large)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 10.7 | |
| Best-of-NModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 9.9 | |
| Self-Certainty (large)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 9.6 | |
| Self-Certainty (large)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 9.6 | |
| Borda count (large)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 9.6 | |
| Borda count (small)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 9.6 | |
| Borda count (small)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 9.2 | |
| Self-Certainty (small)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 9.2 | |
| Self-Certainty (small)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 8.8 | |
| Majority@kModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 8.5 | |
| Borda count (small)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 8.5 | |
| Borda count (large)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 8.5 | |
| Majority@kModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 8.1 | |
| Best-of-NModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 7.7 | |
| Small greedyModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 7.4 | |
| Small greedyModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 7.4 | |
| Small greedyModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 7.4 | |
| Majority@kModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 7.4 | |
| Self-Certainty (small)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 7 |