Mathematical Reasoning on AIME24 v1 (test)
25.6AccuracyPass@k
Evaluation results
| Method | Links | |
|---|---|---|
| Pass@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 25.6 | |
| Large greedyModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 20 | |
| Large greedyModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 20 | |
| Large greedyModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 20 | |
| Pass@kModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 17.8 | |
| Large greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 16.7 | |
| Large greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 16.7 | |
| Pass@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 16.7 | |
| Large greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 16.7 | |
| CGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=32, Chunk Length (L)=202026.06 | 13.3 | |
| LGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=32, Chunk Length (L)=202026.06 | 12.2 | |
| CGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=32, Chunk Length (L)=202026.06 | 12.2 | |
| PRM guided searchModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 11.1 | |
| CGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=16, Chunk Length (L)=202026.06 | 10 | |
| LGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=16, Chunk Length (L)=202026.06 | 8.9 | |
| CGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=16, Chunk Length (L)=202026.06 | 8.9 | |
| Self-Certainty (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 8.9 | |
| Pass@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 7.8 | |
| CGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=8, Chunk Length (L)=202026.06 | 7.8 | |
| Pass@kModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 7.8 | |
| LGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=32, Chunk Length (L)=202026.06 | 7.8 | |
| PRM guided searchModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 6.7 | |
| PRM guided searchModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 5.6 | |
| Borda count (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 5.6 | |
| Borda count (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 5.6 | |
| PRM guided searchModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 5.6 | |
| Self-Certainty (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 4.4 | |
| Borda count (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 4.4 | |
| Borda count (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 4.4 | |
| Self-Certainty (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 4.4 | |
| Majority@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 4.4 | |
| LGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=16, Chunk Length (L)=202026.06 | 4.4 | |
| Self-Certainty (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 3.3 | |
| Borda count (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 3.3 | |
| Best-of-NModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 3.3 | |
| LGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=8, Chunk Length (L)=202026.06 | 3.3 | |
| Majority@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 3.3 | |
| Self-Certainty (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 3.3 | |
| Self-Certainty (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 3.3 | |
| Borda count (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 3.3 | |
| Best-of-NModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 3.3 | |
| Pass@kModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 3.3 | |
| Self-Certainty (small)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 3.3 | |
| LGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=8, Chunk Length (L)=202026.06 | 3.3 | |
| Self-Certainty (large)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 3.3 | |
| Borda count (large)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 3.3 | |
| Majority@kModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 3.3 | |
| Majority@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 2.2 | |
| Majority@kModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 2.2 | |
| Borda count (small)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 2.2 | |
| Borda count (large)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 2.2 | |
| Best-of-NModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 2.2 | |
| CGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=8, Chunk Length (L)=202026.06 | 2.2 | |
| PRM guided searchModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 2.2 | |
| Borda count (small)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 2.2 | |
| Self-Certainty (large)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 2.2 | |
| Borda count (large)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 2.2 | |
| Self-Certainty (large)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 1.1 | |
| Majority@kModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 1.1 | |
| Borda count (small)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 1.1 | |
| Best-of-NModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 1.1 | |
| Self-Certainty (small)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 1.1 | |
| Small greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.06 | 0 | |
| Small greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 0 | |
| Best-of-NModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.06 | 0 | |
| Small greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.06 | 0 | |
| Small greedyModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 0 | |
| PRM guided searchModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.06 | 0 | |
| Small greedyModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 0 | |
| Self-Certainty (small)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.06 | 0 | |
| Small greedyModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 0 | |
| Best-of-NModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.06 | 0 |