Mathematical Reasoning on GSM8K (test) (Accuracy, #Token (k))
89.2AccuracyCARD
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| CARDBackbone=Qwen3-1.7B2025.12 | 89.2 | — | — | 921 | |
| Qwen3-4B-Inst-2507Chat Template=Off, Decoding=Static, k=12026.02 | 89.1 | 1 | — | — | |
| Qwen3-4B-Inst-2507Chat Template=Off, Decoding=ConfAdapt, tau=0.9952026.02 | 88.2 | 2.1 | — | — | |
| RDoLTBackbone=Qwen3-1.7B, Strategy=3-tier2025.12 | 87.9 | — | — | 1,987 | |
| Qwen3-4B-Inst-2507Chat Template=Off, Decoding=ConfAdapt, tau=0.992026.02 | 87.6 | 2.2 | — | — | |
| Qwen3-4B-Inst-2507Chat Template=Off, Decoding=ConfAdapt, tau=0.982026.02 | 87.5 | 2.4 | — | — | |
| Qwen3-4B-Inst-2507Chat Template=Off, Decoding=ConfAdapt, tau=0.972026.02 | 87.5 | 2.5 | — | — | |
| Zero-ShotBackbone=Qwen3-1.7B2025.12 | 87.5 | — | — | 1,798 | |
| Qwen3-4B-Inst-2507Chat Template=Off, Decoding=ConfAdapt, tau=0.962026.02 | 87 | 2.6 | — | — | |
| Qwen3-4B-Inst-2507Chat Template=Off, Decoding=ConfAdapt, tau=0.952026.02 | 86.9 | 2.7 | — | — | |
| ToTBackbone=Qwen3-1.7B, Search strategy=beam-32025.12 | 86.8 | — | — | 1,542 | |
| CARDBackbone=DeepSeek-R1-Distill-Qwen-1.5B2025.12 | 85.8 | — | — | 1,043 | |
| MCTS w/ Var Reducesearch_strategy=MCTS, variant=Var Reduce2025.02 | 85.4 | 8.84 | — | — | |
| MCTS w/ FETCHsearch_strategy=MCTS, variant=FETCH2025.02 | 85.3 | 2.6 | — | — | |
| BFS w/ FETCHsearch_strategy=BFS, variant=FETCH2025.02 | 85.2 | 0.87 | — | — | |
| CoTBackbone=Qwen3-1.7B2025.12 | 85.1 | — | — | 816 | |
| BFS w/ Var Reducesearch_strategy=BFS, variant=Var Reduce2025.02 | 84.7 | 2.25 | — | — | |
| Beam Search w/ Var Reducesearch_strategy=Beam Search, variant=Var Reduce2025.02 | 84.6 | 1.58 | — | — | |
| MCTS w/ State Mergesearch_strategy=MCTS, variant=State Merge2025.02 | 84.5 | 3.03 | — | — | |
| Beam Search w/ FETCHsearch_strategy=Beam Search, variant=FETCH2025.02 | 84.2 | 1.19 | — | — | |
| RDoLTBackbone=DeepSeek-R1-Distill-Qwen-1.5B, Strategy=3-tier2025.12 | 84.1 | — | — | 2,289 | |
| Tree Search w/ Var Reducesearch_strategy=Tree Search, variant=Var Reduce2025.02 | 83.8 | 0.53 | — | — | |
| MCTSsearch_strategy=MCTS2025.02 | 83.8 | 10.7 | — | — | |
| Tree Search w/ FETCHsearch_strategy=Tree Search, variant=FETCH2025.02 | 83.7 | 0.46 | — | — | |
| Zero-ShotBackbone=DeepSeek-R1-Distill-Qwen-1.5B2025.12 | 83.65 | — | — | 2,043 | |
| BFS w/ State Mergesearch_strategy=BFS, variant=State Merge2025.02 | 83.6 | 0.88 | — | — | |
| Qwen3-4B-Inst-2507Chat Template=Off, Decoding=ConfAdapt, tau=0.92026.02 | 83.6 | 3.1 | — | — | |
| Beam Search w/ State Mergesearch_strategy=Beam Search, variant=State Merge2025.02 | 83.2 | 1.17 | — | — | |
| ToTBackbone=DeepSeek-R1-Distill-Qwen-1.5B, Search strategy=beam-32025.12 | 82.9 | — | — | 3,187 | |
| Beam Searchsearch_strategy=Beam Search2025.02 | 82.8 | 1.55 | — | — | |
| Tree Search w/ State Mergesearch_strategy=Tree Search, variant=State Merge2025.02 | 82.5 | 0.48 | — | — | |
| BFSsearch_strategy=BFS2025.02 | 82.4 | 2.47 | — | — | |
| Qwen3-4B-Inst-2507Chat Template=Off, Decoding=Static, k=22026.02 | 82.3 | 2 | — | — | |
| Qwen3-4B-Inst-2507Chat Template=Off, Decoding=ConfAdapt, tau=0.872026.02 | 82.3 | 3.2 | — | — | |
| Best-of-Nsearch_strategy=Best-of-N2025.02 | 81.9 | 0.83 | — | — | |
| Tree Searchsearch_strategy=Tree Search2025.02 | 81.8 | 0.55 | — | — | |
| Weighted Votingsearch_strategy=Weighted Voting2025.02 | 81.7 | 0.83 | — | — | |
| CARDBackbone=Qwen3-0.6B2025.12 | 81.4 | — | — | 897 | |
| Qwen3-4B-Inst-2507Chat Template=Off, Decoding=ConfAdapt, tau=0.852026.02 | 81.3 | 3.3 | — | — | |
| CoTBackbone=DeepSeek-R1-Distill-Qwen-1.5B2025.12 | 81.2 | — | — | 2,066 | |
| RDoLTBackbone=Qwen3-0.6B, Strategy=3-tier2025.12 | 79.8 | — | — | 2,156 | |
| Zero-ShotBackbone=Qwen3-0.6B2025.12 | 79.2 | — | — | 1,687 | |
| TALE-PT-SFTLLM=Llama-3.1-8B-Instruct2024.12 | 78.57 | — | 139.63 | — | |
| ToTBackbone=Qwen3-0.6B, Search strategy=beam-32025.12 | 78.4 | — | — | 2,934 | |
| Qwen3-4B-Inst-2507Chat Template=Off, Decoding=ConfAdapt, tau=0.82026.02 | 77.6 | 3.6 | — | — | |
| Vanilla CoTLLM=Llama-3.1-8B-Instruct2024.12 | 77.56 | — | 241.51 | — | |
| CoTBackbone=Qwen3-0.6B2025.12 | 76.8 | — | — | 1,847 | |
| Qwen3-4B-Inst-2507Chat Template=Off, Decoding=Baseline, Step=0, k=12026.02 | 75.4 | 1 | — | — | |
| Self-Consistencysearch_strategy=Self-Consistency2025.02 | 75.3 | 0.83 | — | — | |
| TALE-PT-DPOLLM=Llama-3.1-8B-Instruct2024.12 | 74.11 | — | 149.93 | — | |
| Qwen3-4B-Inst-2507Chat Template=Off, Decoding=ConfAdapt, tau=0.752026.02 | 73.2 | 3.8 | — | — | |
| L3.1-8B-MagpieChat Template=On, Decoding=Baseline, Step=0, k=12026.02 | 69.5 | 1 | — | — | |
| Qwen3-4B-Inst-2507Chat Template=Off, Decoding=ConfAdapt, tau=0.72026.02 | 68.2 | 4.1 | — | — | |
| L3.1-8B-MagpieChat Template=On, Decoding=Static, k=12026.02 | 66 | 1 | — | — | |
| L3.1-8B-MagpieChat Template=On, Decoding=ConfAdapt, tau=0.992026.02 | 66 | 2.1 | — | — | |
| Greedy Decodingsearch_strategy=Greedy Decoding2025.02 | 65.7 | 0.08 | — | — | |
| L3.1-8B-MagpieChat Template=On, Decoding=ConfAdapt, tau=0.9952026.02 | 65.7 | 1.9 | — | — | |
| L3.1-8B-MagpieChat Template=On, Decoding=ConfAdapt, tau=0.982026.02 | 65.5 | 2.3 | — | — | |
| L3.1-8B-MagpieChat Template=On, Decoding=ConfAdapt, tau=0.972026.02 | 65.3 | 2.5 | — | — | |
| L3.1-8B-MagpieChat Template=On, Decoding=ConfAdapt, tau=0.962026.02 | 64.9 | 2.6 | — | — | |
| L3.1-8B-MagpieChat Template=On, Decoding=ConfAdapt, tau=0.952026.02 | 64.7 | 2.8 | — | — | |
| L3.1-8B-MagpieChat Template=On, Decoding=ConfAdapt, tau=0.92026.02 | 64.1 | 3.3 | — | — | |
| L3.1-8B-MagpieChat Template=On, Decoding=ConfAdapt, tau=0.872026.02 | 62.3 | 3.6 | — | — | |
| Qwen3-4B-Inst-2507Chat Template=Off, Decoding=ConfAdapt, tau=0.652026.02 | 62 | 4.4 | — | — | |
| Qwen3-4B-Inst-2507Chat Template=Off, Decoding=Static, k=32026.02 | 61.6 | 3 | — | — | |
| L3.1-8B-MagpieChat Template=On, Decoding=ConfAdapt, tau=0.852026.02 | 61.2 | 3.7 | — | — | |
| L3.1-8B-MagpieChat Template=On, Decoding=Static, k=22026.02 | 60.7 | 2 | — | — | |
| L3.1-8B-MagpieChat Template=On, Decoding=ConfAdapt, tau=0.82026.02 | 59.2 | 4.2 | — | — | |
| Qwen3-4B-Inst-2507Chat Template=Off, Decoding=ConfAdapt, tau=0.62026.02 | 58.8 | 4.6 | — | — | |
| L3.1-8B-MagpieChat Template=On, Decoding=ConfAdapt, tau=0.752026.02 | 56.9 | 4.6 | — | — | |
| L3.1-8B-MagpieChat Template=On, Decoding=ConfAdapt, tau=0.72026.02 | 55 | 5.1 | — | — | |
| L3.1-8B-MagpieChat Template=On, Decoding=Static, k=32026.02 | 53 | 3 | — | — | |
| L3.1-8B-MagpieChat Template=On, Decoding=ConfAdapt, tau=0.652026.02 | 51.5 | 5.6 | — | — | |
| L3.1-8B-MagpieChat Template=On, Decoding=Static, k=42026.02 | 49.8 | 4 | — | — | |
| L3.1-8B-MagpieChat Template=On, Decoding=ConfAdapt, tau=0.62026.02 | 46.6 | 6.1 | — | — | |
| Qwen3-4B-Inst-2507Chat Template=Off, Decoding=Static, k=42026.02 | 44.2 | 4 | — | — | |
| L3.1-8B-MagpieChat Template=On, Decoding=Static, k=52026.02 | 41.5 | 5 | — | — | |
| Qwen3-4B-Inst-2507Chat Template=Off, Decoding=Static, k=52026.02 | 28.9 | 5 | — | — | |
| Directly AnsweringLLM=Llama-3.1-8B-Instruct2024.12 | 21 | — | 38.54 | — |