Mathematical Reasoning on MultiArith (Accuracy, Reasoning Length)
97.18AccuracyLTF
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| LTFBackbone=LLaMA-3.1 Instruct 8B2026.06 | 97.18 | 2.16 | |
| ReGuLaRBackbone=LLaMA-3.1 Instruct 8B2026.06 | 95.28 | 2.26 | |
| CoLaRBackbone=LLaMA-3.1 Instruct 8B2026.06 | 93.8 | 3.14 | |
| LTFBackbone=LLaMA-3.2 Instruct 3B2026.06 | 92.94 | 2.1 | |
| ReGuLaRBackbone=LLaMA-3.2 Instruct 3B2026.06 | 91.27 | 2.17 | |
| LTFBackbone=DeepSeek-R1 Distill-Qwen 1.5B2026.06 | 91.1 | 2.21 | |
| LTFBackbone=LLaMA-3.2 Instruct 1B2026.06 | 90.37 | 2.17 | |
| CoLaRBackbone=LLaMA-3.2 Instruct 3B2026.06 | 90.1 | 3.04 | |
| ReGuLaRBackbone=DeepSeek-R1 Distill-Qwen 1.5B2026.06 | 89.04 | 2.29 | |
| ReGuLaRBackbone=LLaMA-3.2 Instruct 1B2026.06 | 88.97 | 2.29 | |
| CoLaRBackbone=DeepSeek-R1 Distill-Qwen 1.5B2026.06 | 87.01 | 3.22 | |
| CoLaRBackbone=LLaMA-3.2 Instruct 1B2026.06 | 86.93 | 3.22 | |
| CoconutBackbone=LLaMA-3.1 Instruct 8B2026.06 | 62.41 | 6 | |
| CoconutBackbone=LLaMA-3.2 Instruct 3B2026.06 | 58.79 | 6 | |
| iCoTBackbone=LLaMA-3.1 Instruct 8B2026.06 | 50.39 | 0 | |
| iCoTBackbone=LLaMA-3.2 Instruct 3B2026.06 | 46.42 | 0 | |
| SoftCoT++Backbone=LLaMA-3.1 Instruct 8B2026.06 | 43.19 | 121.98 | |
| CoconutBackbone=DeepSeek-R1 Distill-Qwen 1.5B2026.06 | 41.31 | 6 | |
| CoconutBackbone=LLaMA-3.2 Instruct 1B2026.06 | 41.3 | 6 | |
| Assist-CoTBackbone=LLaMA-3.1 Instruct 8B2026.06 | 40.6 | 122.64 | |
| SoftCoT++Backbone=LLaMA-3.2 Instruct 3B2026.06 | 40.26 | 121.74 | |
| CoTBackbone=LLaMA-3.1 Instruct 8B2026.06 | 38.82 | 121.84 | |
| iCoTBackbone=DeepSeek-R1 Distill-Qwen 1.5B2026.06 | 38.24 | 0 | |
| iCoTBackbone=LLaMA-3.2 Instruct 1B2026.06 | 38.19 | 0 | |
| Assist-CoTBackbone=LLaMA-3.2 Instruct 3B2026.06 | 37.03 | 121.98 | |
| CoTBackbone=LLaMA-3.2 Instruct 3B2026.06 | 35.16 | 121.54 | |
| SoftCoT++Backbone=DeepSeek-R1 Distill-Qwen 1.5B2026.06 | 32.18 | 121.92 | |
| SoftCoT++Backbone=LLaMA-3.2 Instruct 1B2026.06 | 32.13 | 120.66 | |
| Assist-CoTBackbone=DeepSeek-R1 Distill-Qwen 1.5B2026.06 | 30.14 | 122.7 | |
| Assist-CoTBackbone=LLaMA-3.2 Instruct 1B2026.06 | 30.06 | 121.86 | |
| CoTBackbone=DeepSeek-R1 Distill-Qwen 1.5B2026.06 | 28.33 | 121.53 | |
| CoTBackbone=LLaMA-3.2 Instruct 1B2026.06 | 28.29 | 120.14 | |
| CODIBackbone=LLaMA-3.2 Instruct 1B2026.06 | 19.2 | 6 |