Mathematical Reasoning on GSM-Plus (test)
68.8AccuracyMCTS w/ FETCH
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MCTS w/ FETCHsearch_strategy=MCTS, variant=FETCH2025.02 | 68.8 | 3.53 | |
| MCTS w/ Var Reducesearch_strategy=MCTS, variant=Var Reduce2025.02 | 68.6 | 14.4 | |
| BFS w/ FETCHsearch_strategy=BFS, variant=FETCH2025.02 | 68.4 | 1.09 | |
| Beam Search w/ FETCHsearch_strategy=Beam Search, variant=FETCH2025.02 | 68.2 | 1.62 | |
| Beam Search w/ Var Reducesearch_strategy=Beam Search, variant=Var Reduce2025.02 | 68 | 1.89 | |
| Tree Search w/ FETCHsearch_strategy=Tree Search, variant=FETCH2025.02 | 67.7 | 0.69 | |
| BFS w/ Var Reducesearch_strategy=BFS, variant=Var Reduce2025.02 | 67.6 | 2.89 | |
| MCTSsearch_strategy=MCTS2025.02 | 67.6 | 14.1 | |
| Beam Search w/ State Mergesearch_strategy=Beam Search, variant=State Merge2025.02 | 67.4 | 1.61 | |
| Tree Search w/ Var Reducesearch_strategy=Tree Search, variant=Var Reduce2025.02 | 67 | 0.89 | |
| MCTS w/ State Mergesearch_strategy=MCTS, variant=State Merge2025.02 | 67 | 3.43 | |
| Beam Searchsearch_strategy=Beam Search2025.02 | 66.9 | 1.88 | |
| BFS w/ State Mergesearch_strategy=BFS, variant=State Merge2025.02 | 66.4 | 1.12 | |
| Tree Searchsearch_strategy=Tree Search2025.02 | 66 | 0.92 | |
| Tree Search w/ State Mergesearch_strategy=Tree Search, variant=State Merge2025.02 | 65.7 | 0.69 | |
| BFSsearch_strategy=BFS2025.02 | 65.5 | 2.84 | |
| Best-of-Nsearch_strategy=Best-of-N2025.02 | 63.4 | 0.95 | |
| Weighted Votingsearch_strategy=Weighted Voting2025.02 | 62 | 0.95 | |
| Self-Consistencysearch_strategy=Self-Consistency2025.02 | 60.1 | 0.95 | |
| Greedy Decodingsearch_strategy=Greedy Decoding2025.02 | 50 | 0.09 | |
| TeacherTeacher and Student Model Pair=Qwen-2.5-3b to Qwen-2.5-0.5b2025.10 | 45.4 | — | |
| TeacherTeacher and Student Model Pair=Llama-3.1-3b to Llama-3.1-1b2025.10 | 38.6 | — | |
| TeacherTeacher and Student Model Pair=Gemma-7b to Gemma-2b2025.10 | 33.4 | — | |
| AdaSwitchTeacher and Student Model Pair=Qwen-2.5-3b to Qwen-2.5-0.5b2025.10 | 21.6 | — | |
| ImitKDTeacher and Student Model Pair=Qwen-2.5-3b to Qwen-2.5-0.5b2025.10 | 21.3 | — | |
| SWITCHTeacher and Student Model Pair=Qwen-2.5-3b to Qwen-2.5-0.5b2025.10 | 20.8 | — | |
| SKDTeacher and Student Model Pair=Qwen-2.5-3b to Qwen-2.5-0.5b2025.10 | 20.4 | — | |
| On-policyTeacher and Student Model Pair=Qwen-2.5-3b to Qwen-2.5-0.5b2025.10 | 20.2 | — | |
| Off-policyTeacher and Student Model Pair=Qwen-2.5-3b to Qwen-2.5-0.5b2025.10 | 17.9 | — | |
| AdaSwitchTeacher and Student Model Pair=Llama-3.1-3b to Llama-3.1-1b2025.10 | 17.3 | — | |
| SKDTeacher and Student Model Pair=Llama-3.1-3b to Llama-3.1-1b2025.10 | 16.9 | — | |
| SWITCHTeacher and Student Model Pair=Llama-3.1-3b to Llama-3.1-1b2025.10 | 16 | — | |
| ImitKDTeacher and Student Model Pair=Llama-3.1-3b to Llama-3.1-1b2025.10 | 15.6 | — | |
| SFTTeacher and Student Model Pair=Qwen-2.5-3b to Qwen-2.5-0.5b2025.10 | 15.5 | — | |
| SeqKDTeacher and Student Model Pair=Qwen-2.5-3b to Qwen-2.5-0.5b2025.10 | 14.4 | — | |
| On-policyTeacher and Student Model Pair=Llama-3.1-3b to Llama-3.1-1b2025.10 | 14.3 | — | |
| AdaSwitchTeacher and Student Model Pair=Gemma-7b to Gemma-2b2025.10 | 14.2 | — | |
| SKDTeacher and Student Model Pair=Gemma-7b to Gemma-2b2025.10 | 13.9 | — | |
| Off-policyTeacher and Student Model Pair=Llama-3.1-3b to Llama-3.1-1b2025.10 | 13.8 | — | |
| On-policyTeacher and Student Model Pair=Gemma-7b to Gemma-2b2025.10 | 13.8 | — | |
| SWITCHTeacher and Student Model Pair=Gemma-7b to Gemma-2b2025.10 | 13.8 | — | |
| ImitKDTeacher and Student Model Pair=Gemma-7b to Gemma-2b2025.10 | 13.5 | — | |
| SFTTeacher and Student Model Pair=Llama-3.1-3b to Llama-3.1-1b2025.10 | 12 | — | |
| SeqKDTeacher and Student Model Pair=Llama-3.1-3b to Llama-3.1-1b2025.10 | 11.8 | — | |
| SFTTeacher and Student Model Pair=Gemma-7b to Gemma-2b2025.10 | 11.7 | — | |
| Off-policyTeacher and Student Model Pair=Gemma-7b to Gemma-2b2025.10 | 10.7 | — | |
| SeqKDTeacher and Student Model Pair=Gemma-7b to Gemma-2b2025.10 | 10.5 | — | |
| StudentTeacher and Student Model Pair=Llama-3.1-3b to Llama-3.1-1b2025.10 | 0 | — | |
| StudentTeacher and Student Model Pair=Qwen-2.5-3b to Qwen-2.5-0.5b2025.10 | 0 | — | |
| StudentTeacher and Student Model Pair=Gemma-7b to Gemma-2b2025.10 | 0 | — |