Mathematical Reasoning on GSM-Hard (Accuracy, Reasoning Length)
53.4AccuracyCoT-SFT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CoT-SFTBackbone=Qwen3-VL-4B-Instruct2026.06 | 53.4 | 191.1 | |
| CoT-SFTBackbone=Qwen3-4B2026.06 | 52.3 | 189.4 | |
| CoT-SFTBackbone=Qwen3-VL-2B-Instruct2026.06 | 33.1 | 207.2 | |
| DLR (Full)Backbone=Qwen3-4B2026.06 | 19.3 | 7 | |
| DLR (Full)Backbone=Qwen3-VL-4B-Instruct2026.06 | 18.3 | 7 | |
| CoT-SFTBackbone=LLaMA-3.2-1B2026.06 | 15.6 | 197.5 | |
| DLR (Full)Backbone=Qwen3-VL-2B-Instruct2026.06 | 15.4 | 6.3 | |
| DLR (SFT)Backbone=Qwen3-VL-4B-Instruct2026.06 | 14.7 | 6.8 | |
| RoTBackbone=Qwen3-VL-4B-Instruct2026.06 | 14.1 | 32 | |
| DLR (SFT)Backbone=Qwen3-4B2026.06 | 13.7 | 6.8 | |
| LTFBackbone=LLaMA-3.1 Instruct 8B2026.06 | 13.19 | 3.7 | |
| ReGuLaRBackbone=LLaMA-3.1 Instruct 8B2026.06 | 12.53 | 3.92 | |
| LTFBackbone=LLaMA-3.2 Instruct 3B2026.06 | 11.88 | 3.64 | |
| ReGuLaRBackbone=LLaMA-3.2 Instruct 3B2026.06 | 11.2 | 3.73 | |
| DLR (Full)Backbone=LLaMA-3.2-1B2026.06 | 11 | 6.6 | |
| CoLaRBackbone=LLaMA-3.1 Instruct 8B2026.06 | 10.97 | 6.53 | |
| CoLaRBackbone=LLaMA-3.2 Instruct 3B2026.06 | 9.73 | 6.46 | |
| Direct-SFTBackbone=Qwen3-VL-4B-Instruct2026.06 | 9.5 | 0 | |
| Direct-SFTBackbone=Qwen3-4B2026.06 | 9.2 | 0 | |
| CoLaRBackbone=Qwen3-4B2026.06 | 9.2 | 47.4 | |
| LTFBackbone=DeepSeek-R1 Distill-Qwen 1.5B2026.06 | 8.83 | 3.94 | |
| LTFBackbone=LLaMA-3.2 Instruct 1B2026.06 | 8.71 | 3.97 | |
| RoTBackbone=Qwen3-VL-2B-Instruct2026.06 | 8.6 | 32 | |
| ReGuLaRBackbone=DeepSeek-R1 Distill-Qwen 1.5B2026.06 | 8.33 | 4.1 | |
| ReGuLaRBackbone=LLaMA-3.2-1B2026.06 | 8.3 | 4.1 | |
| CoconutBackbone=LLaMA-3.1 Instruct 8B2026.06 | 8.26 | 6 | |
| ReGuLaRBackbone=LLaMA-3.2 Instruct 1B2026.06 | 8.25 | 4.1 | |
| CoLaRBackbone=Qwen3-VL-2B-Instruct2026.06 | 8.2 | 6.5 | |
| CoLaRBackbone=Qwen3-VL-4B-Instruct2026.06 | 8.2 | 47.8 | |
| CoconutBackbone=LLaMA-3.2 Instruct 3B2026.06 | 7.34 | 6 | |
| CoLaRBackbone=DeepSeek-R1 Distill-Qwen 1.5B2026.06 | 6.32 | 7.03 | |
| CoLaRBackbone=LLaMA-3.2 Instruct 1B2026.06 | 6.22 | 7.03 | |
| CoLaRBackbone=LLaMA-3.2-1B2026.06 | 6.2 | 7 | |
| SoftCoT++Backbone=LLaMA-3.1 Instruct 8B2026.06 | 5.91 | 124.82 | |
| iCoTBackbone=LLaMA-3.1 Instruct 8B2026.06 | 5.82 | 0 | |
| CoconutBackbone=Qwen3-4B2026.06 | 5.4 | 6 | |
| SoftCoT++Backbone=LLaMA-3.2 Instruct 3B2026.06 | 5.21 | 124.72 | |
| CoconutBackbone=Qwen3-VL-4B-Instruct2026.06 | 5.2 | 6 | |
| iCoTBackbone=LLaMA-3.2 Instruct 3B2026.06 | 5.19 | 0 | |
| CoconutBackbone=DeepSeek-R1 Distill-Qwen 1.5B2026.06 | 4.96 | 6 | |
| CoconutBackbone=LLaMA-3.2-1B2026.06 | 4.9 | 6 | |
| CoconutBackbone=LLaMA-3.2 Instruct 1B2026.06 | 4.87 | 6 | |
| Assist-CoTBackbone=LLaMA-3.1 Instruct 8B2026.06 | 4.75 | 125.59 | |
| Direct-SFTBackbone=Qwen3-VL-2B-Instruct2026.06 | 4.7 | 0 | |
| CoTBackbone=LLaMA-3.1 Instruct 8B2026.06 | 4.51 | 121.66 | |
| CoconutBackbone=Qwen3-VL-2B-Instruct2026.06 | 4.2 | 6 | |
| Assist-CoTBackbone=LLaMA-3.2 Instruct 3B2026.06 | 4.16 | 125.03 | |
| Direct-SFTBackbone=LLaMA-3.2-1B2026.06 | 4.1 | 0 | |
| iCoTBackbone=Qwen3-4B2026.06 | 4.1 | 0 | |
| iCoTBackbone=LLaMA-3.2-1B2026.06 | 3.9 | 0 | |
| SoftCoT++Backbone=DeepSeek-R1 Distill-Qwen 1.5B2026.06 | 3.86 | 123.72 | |
| CoTBackbone=LLaMA-3.2 Instruct 3B2026.06 | 3.84 | 121.68 | |
| iCoTBackbone=DeepSeek-R1 Distill-Qwen 1.5B2026.06 | 3.83 | 0 | |
| iCoTBackbone=LLaMA-3.2 Instruct 1B2026.06 | 3.82 | 0 | |
| SoftCoT++Backbone=LLaMA-3.2 Instruct 1B2026.06 | 3.77 | 123.56 | |
| Assist-CoTBackbone=DeepSeek-R1 Distill-Qwen 1.5B2026.06 | 3.07 | 126.03 | |
| Assist-CoTBackbone=LLaMA-3.2 Instruct 1B2026.06 | 3.02 | 125.19 | |
| CODIBackbone=LLaMA-3.2-1B2026.06 | 3 | 6 | |
| CODIBackbone=LLaMA-3.2 Instruct 1B2026.06 | 2.94 | 6 | |
| CoTBackbone=DeepSeek-R1 Distill-Qwen 1.5B2026.06 | 2.81 | 120.78 | |
| CoTBackbone=LLaMA-3.2 Instruct 1B2026.06 | 2.74 | 120.46 | |
| CODIBackbone=Qwen3-VL-2B-Instruct2026.06 | 2.4 | 6 | |
| CODIBackbone=Qwen3-4B2026.06 | 2.2 | 6 | |
| CODIBackbone=Qwen3-VL-4B-Instruct2026.06 | 2 | 6 |