Mathematical Reasoning on MATH v1 (test)
83.8AccuracyPass@k
Evaluation results
| Method | Links | |
|---|---|---|
| Pass@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 83.8 | |
| Pass@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 78.6 | |
| Large greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 78 | |
| Large greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 78 | |
| Large greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 78 | |
| Pass@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 71 | |
| PRM guided searchModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 69.8 | |
| CGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=32, Chunk Length (L)=202026.06 | 68.8 | |
| Large greedyModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 68 | |
| Large greedyModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 68 | |
| Large greedyModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 68 | |
| LGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=32, Chunk Length (L)=202026.06 | 67.4 | |
| PRM guided searchModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 66.4 | |
| Pass@kModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 66.4 | |
| CGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=16, Chunk Length (L)=202026.06 | 64.6 | |
| PRM guided searchModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 64.2 | |
| Borda count (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 63.4 | |
| CGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=8, Chunk Length (L)=202026.06 | 63.2 | |
| Borda count (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 63.2 | |
| Majority@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 63 | |
| LGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=16, Chunk Length (L)=202026.06 | 61.8 | |
| Pass@kModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 60.8 | |
| LGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=8, Chunk Length (L)=202026.06 | 60 | |
| Borda count (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 60 | |
| Majority@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 59.6 | |
| Borda count (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 59.6 | |
| Borda count (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 57.6 | |
| Borda count (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 57 | |
| Majority@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 56.6 | |
| Best-of-NModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 55.8 | |
| Self-Certainty (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 53.6 | |
| Best-of-NModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 53.4 | |
| Qwen2-7BInstruction-tuned=true2024.07 | 52.9 | |
| Small greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 52.8 | |
| Small greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 52.8 | |
| Small greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 52.8 | |
| Self-Certainty (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 52.6 | |
| Self-Certainty (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 52.6 | |
| Best-of-NModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 52.6 | |
| Self-Certainty (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 51.2 | |
| Self-Certainty (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 51 | |
| GLM-4-9BInstruction-tuned=true2024.07 | 50.6 | |
| Pass@kModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 49.6 | |
| Self-Certainty (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 49.2 | |
| Yi-1.5-9BInstruction-tuned=true2024.07 | 47.7 | |
| PRM guided searchModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 46.8 | |
| CGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=32, Chunk Length (L)=202026.06 | 46.4 | |
| CGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=16, Chunk Length (L)=202026.06 | 44.4 | |
| LGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=32, Chunk Length (L)=202026.06 | 44.2 | |
| PRM guided searchModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 43.4 | |
| PRM guided searchModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 40.2 | |
| CGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=8, Chunk Length (L)=202026.06 | 39.8 | |
| LGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=16, Chunk Length (L)=202026.06 | 38.8 | |
| Borda count (small)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 37.8 | |
| Borda count (large)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 37.8 | |
| LGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=8, Chunk Length (L)=202026.06 | 36.6 | |
| Majority@kModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 36 | |
| Borda count (large)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 34.6 | |
| Majority@kModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 34.2 | |
| Borda count (small)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 33.4 | |
| Self-Certainty (large)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 33.4 | |
| Best-of-NModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 32.2 | |
| Borda count (large)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 31.4 | |
| Best-of-NModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 30.6 | |
| Llama-3-8BInstruction-tuned=true2024.07 | 30 | |
| Self-Certainty (large)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 30 | |
| Self-Certainty (large)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 29.8 | |
| Best-of-NModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 29 | |
| Borda count (small)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 28.6 | |
| Majority@kModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 28.2 | |
| Self-Certainty (small)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 25.8 | |
| Small greedyModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 25.2 | |
| Small greedyModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 25.2 | |
| Small greedyModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 25.2 | |
| Self-Certainty (small)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 24.4 | |
| Qwen1.5-7BInstruction-tuned=true2024.07 | 23.2 | |
| Self-Certainty (small)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 23 |