Mathematical Reasoning on SVAMP (Accuracy, Reasoning Length)
72.29AccuracyLTF
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LTFBackbone=LLaMA-3.1 Instruct 8B2026.06 | 72.29 | 2.09 | |
| ReGuLaRBackbone=LLaMA-3.1 Instruct 8B2026.06 | 68.54 | 2.07 | |
| LTFBackbone=LLaMA-3.2 Instruct 3B2026.06 | 65.48 | 2.14 | |
| ReGuLaRBackbone=LLaMA-3.2 Instruct 3B2026.06 | 62.32 | 2.23 | |
| CoLaRBackbone=LLaMA-3.1 Instruct 8B2026.06 | 61.3 | 2.83 | |
| CoLaRBackbone=LLaMA-3.2 Instruct 3B2026.06 | 60.7 | 2.72 | |
| CoconutBackbone=LLaMA-3.1 Instruct 8B2026.06 | 57.18 | 6 | |
| LTFBackbone=DeepSeek-R1 Distill-Qwen 1.5B2026.06 | 52.76 | 2.08 | |
| CoconutBackbone=LLaMA-3.2 Instruct 3B2026.06 | 52.47 | 6 | |
| LTFBackbone=LLaMA-3.2 Instruct 1B2026.06 | 52.01 | 2.11 | |
| iCoTBackbone=LLaMA-3.1 Instruct 8B2026.06 | 49.48 | 0 | |
| ReGuLaRBackbone=DeepSeek-R1 Distill-Qwen 1.5B2026.06 | 48.2 | 2.1 | |
| ReGuLaRBackbone=LLaMA-3.2 Instruct 1B2026.06 | 48.19 | 2.1 | |
| CoLaRBackbone=DeepSeek-R1 Distill-Qwen 1.5B2026.06 | 47.27 | 2.94 | |
| CoLaRBackbone=LLaMA-3.2 Instruct 1B2026.06 | 47.22 | 2.94 | |
| iCoTBackbone=LLaMA-3.2 Instruct 3B2026.06 | 44.98 | 0 | |
| SoftCoT++Backbone=LLaMA-3.1 Instruct 8B2026.06 | 42.85 | 123.82 | |
| CoconutBackbone=DeepSeek-R1 Distill-Qwen 1.5B2026.06 | 38.98 | 6 | |
| CoconutBackbone=LLaMA-3.2 Instruct 1B2026.06 | 38.9 | 6 | |
| SoftCoT++Backbone=LLaMA-3.2 Instruct 3B2026.06 | 38.89 | 122.96 | |
| Assist-CoTBackbone=LLaMA-3.1 Instruct 8B2026.06 | 38.62 | 124.62 | |
| CoTBackbone=LLaMA-3.1 Instruct 8B2026.06 | 37.43 | 121.56 | |
| iCoTBackbone=DeepSeek-R1 Distill-Qwen 1.5B2026.06 | 36.49 | 0 | |
| iCoTBackbone=LLaMA-3.2 Instruct 1B2026.06 | 36.42 | 0 | |
| Assist-CoTBackbone=LLaMA-3.2 Instruct 3B2026.06 | 35.06 | 123.42 | |
| CoTBackbone=LLaMA-3.2 Instruct 3B2026.06 | 34.01 | 121.52 | |
| SoftCoT++Backbone=DeepSeek-R1 Distill-Qwen 1.5B2026.06 | 31.02 | 123.26 | |
| SoftCoT++Backbone=LLaMA-3.2 Instruct 1B2026.06 | 30.96 | 123.42 | |
| Assist-CoTBackbone=DeepSeek-R1 Distill-Qwen 1.5B2026.06 | 28.12 | 123.76 | |
| Assist-CoTBackbone=LLaMA-3.2 Instruct 1B2026.06 | 28.04 | 123.02 | |
| CoTBackbone=DeepSeek-R1 Distill-Qwen 1.5B2026.06 | 27.23 | 121.48 | |
| CoTBackbone=LLaMA-3.2 Instruct 1B2026.06 | 27.17 | 120.8 | |
| CODIBackbone=LLaMA-3.2 Instruct 1B2026.06 | 21.65 | 6 |