Mathematical Problem Solving on AIME 2024 (test)
28.9AccuracyBudgetThinker
Evaluation Results
| Method | Links | |
|---|---|---|
| BudgetThinkerGeneration Budget (B)=10000, Model Scale=1.5B2025.08 | 28.9 | |
| OriginalGeneration Budget (B)=10000, Model Scale=1.5B2025.08 | 28.88 | |
| ThinkPruneGeneration Budget (B)=10000, Model Scale=1.5B2025.08 | 27.8 | |
| OriginalGeneration Budget (B)=6000, Model Scale=1.5B2025.08 | 25.71 | |
| BudgetThinkerGeneration Budget (B)=6000, Model Scale=1.5B2025.08 | 25.63 | |
| ThinkPruneGeneration Budget (B)=6000, Model Scale=1.5B2025.08 | 25.22 | |
| BudgetThinkerGeneration Budget (B)=4000, Model Scale=1.5B2025.08 | 23.8 | |
| OriginalGeneration Budget (B)=4000, Model Scale=1.5B2025.08 | 23.28 | |
| ThinkPruneGeneration Budget (B)=4000, Model Scale=1.5B2025.08 | 22.78 | |
| BudgetThinkerGeneration Budget (B)=2000, Model Scale=1.5B2025.08 | 16.25 | |
| ThinkPruneGeneration Budget (B)=2000, Model Scale=1.5B2025.08 | 15.22 | |
| OriginalGeneration Budget (B)=2000, Model Scale=1.5B2025.08 | 14.8 |