Mathematical Reasoning on GSM8k Aug
81.2AccuracySFT-CoT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SFT-CoTModel=Qwen3-VL-4B-Instruct2026.01 | 81.2 | 127.3 | |
| Explicit CoTBackbone=LLAMA-3.2-3B, Training Data=GSM8k-Aug2026.06 | 71.5 | — | |
| LOTUSBackbone=LLAMA-3.2-3B, Training Data=GSM8k-Aug2026.06 | 70 | — | |
| Latent-GRPOBase Model=LLaMA-3.2-1B-Instruct, Reasoning Mode=Latent Reasoning (No Sampling)2026.04 | 66.29 | 23.8 | |
| KaVaBackbone=LLAMA-3.2-3B, Training Data=GSM8k-Aug2026.06 | 65.7 | — | |
| GRPOBase Model=LLaMA-3.2-1B-Instruct, Reasoning Mode=Explicit Reasoning2026.04 | 62.26 | 111.1 | |
| SFT-CoTModel=Qwen3-VL-2B-Instruct2026.01 | 59.7 | 131.4 | |
| Explicit CoTBackbone=LLAMA-3.2-1B, Training Data=GSM8k-Aug2026.06 | 58.4 | — | |
| LOTUSBackbone=LLAMA-3.2-1B, Training Data=GSM8k-Aug2026.06 | 57.3 | — | |
| KaVaBackbone=LLAMA-3.2-1B, Training Data=GSM8k-Aug2026.06 | 56.5 | — | |
| PCCoTBackbone=LLAMA-3.2-3B, Training Data=GSM8k-Aug2026.06 | 54.7 | — | |
| PCCoTBackbone=LLAMA-3.2-1B, Training Data=GSM8k-Aug2026.06 | 54.2 | — | |
| LTFModel=LLaMA-3.1 Instruct 8B, Finetuning Setting=true2026.06 | 53.14 | 3.37 | |
| ReGuLaRModel=LLaMA-3.1 Instruct 8B, Finetuning Setting=true2026.06 | 50.14 | 3.93 | |
| SFTBase Model=LLaMA-3.2-1B-Instruct, Reasoning Mode=Explicit Reasoning2026.04 | 49.51 | 77.46 | |
| PCCoTBackbone=GPT-2, Training Data=GSM8k-Aug2026.06 | 49.5 | — | |
| CoT2026.02 | 49.4 | 25.6 | |
| CoT2026.02 | 49.4 | 25.6 | |
| Soft-GRPOBase Model=LLaMA-3.2-1B-Instruct, Reasoning Mode=Latent Reasoning (No Sampling)2026.04 | 48.34 | 21.6 | |
| LTFModel=LLaMA-3.2 Instruct 3B, Finetuning Setting=true2026.06 | 48.3 | 3.42 | |
| Latent-SFTBase Model=LLaMA-3.2-1B-Instruct, Reasoning Mode=Latent Reasoning (No Sampling)2026.04 | 47.55 | 22 | |
| ReGuLaRModel=LLaMA-3.2 Instruct 3B, Finetuning Setting=true2026.06 | 45.59 | 3.89 | |
| CoLTSeed count=22026.02 | 45.5 | 10.84 | |
| CoLTSeed count=12026.02 | 45.3 | 7.73 | |
| CoLaRModel=LLaMA-3.1 Instruct 8B, Finetuning Setting=true2026.06 | 45.19 | 5.62 | |
| Coconut + SIM-CoTBackbone=GPT-2, Training Data=GSM8k-Aug2026.06 | 44.8 | — | |
| LOTUSBackbone=GPT-2, Training Data=GSM8k-Aug2026.06 | 44.1 | — | |
| Explicit CoTBackbone=GPT-2, Training Data=GSM8k-Aug2026.06 | 42.7 | — | |
| CoLaRModel=LLaMA-3.2 Instruct 3B, Finetuning Setting=true2026.06 | 42.58 | 5.48 | |
| SIM-CoT2026.02 | 42.2 | 13.2 | |
| Coconut + SIM-CoTBackbone=LLAMA-3.2-1B, Training Data=GSM8k-Aug2026.06 | 42.2 | — | |
| LSTRCompression ratio=22026.02 | 40.2 | 13.18 | |
| CoLaRCompression ratio=22026.02 | 40.1 | 12.7 | |
| COLARMultiplier=2x2026.02 | 40.1 | 12.7 | |
| SFT-CoTModel=LLaVa-V1.6-Mistral-7B2026.01 | 38.6 | 151.3 | |
| RoT (Ours)Model=Qwen3-VL-4B-Instruct2026.01 | 37.8 | 32 | |
| LTFModel=LLaMA-3.2 Instruct 1B, Finetuning Setting=true2026.06 | 37.09 | 3.34 | |
| LTFModel=DeepSeek-R1-Distill-Qwen-1.5B, Finetuning Setting=true2026.06 | 36.94 | 3.27 | |
| CoconutBackbone=GPT-2, Training Data=GSM8k-Aug2026.06 | 36.6 | — | |
| CoconutModel=LLaMA-3.1 Instruct 8B, Finetuning Setting=true2026.06 | 36.1 | 6 | |
| LSTRCompression ratio=2, Ablation=w/o Skip2026.02 | 35.3 | 13.14 | |
| ReGuLaRBackbone=LLaMA-3.2-1B-Instruct2026.01 | 34.9 | 3.69 | |
| ReGuLaRModel=DeepSeek-R1-Distill-Qwen-1.5B, Finetuning Setting=true2026.06 | 34.69 | 3.54 | |
| ReGuLaRModel=LLaMA-3.2 Instruct 1B, Finetuning Setting=true2026.06 | 34.58 | 3.69 | |
| LSTRCompression ratio=2, Ablation=w/o Sparse2026.02 | 34.2 | 13.43 | |
| CoconutBackbone=LLAMA-3.2-1B, Training Data=GSM8k-Aug2026.06 | 33.2 | — | |
| CoconutModel=LLaMA-3.2 Instruct 3B, Finetuning Setting=true2026.06 | 32.71 | 6 | |
| LSTRCompression ratio=52026.02 | 29.5 | 5.69 | |
| LSTRCompression ratio=5, Ablation=w/o Skip2026.02 | 27.5 | 5.75 | |
| CoLaRModel=DeepSeek-R1-Distill-Qwen-1.5B, Finetuning Setting=true2026.06 | 27.38 | 5.71 | |
| CoLaRCompression ratio=52026.02 | 26.8 | 5.57 | |
| COLARMultiplier=5x2026.02 | 26.8 | 5.57 | |
| CoLaRBackbone=LLaMA-3.2-1B-Instruct, Compression Rate=52026.01 | 26.6 | 5.63 | |
| CoLaRModel=LLaMA-3.2 Instruct 1B, Finetuning Setting=true2026.06 | 26.6 | 5.63 | |
| SFT-w/o CoTModel=Qwen3-VL-4B-Instruct2026.01 | 26.2 | 0 | |
| CoconutModel=DeepSeek-R1-Distill-Qwen-1.5B, Finetuning Setting=true2026.06 | 24.91 | 6 | |
| iCoTModel=LLaMA-3.1 Instruct 8B, Finetuning Setting=true2026.06 | 24.24 | 0 | |
| RoT (Ours)Model=Qwen3-VL-2B-Instruct2026.01 | 23.3 | 32 | |
| Coconut2026.02 | 23.1 | 6 | |
| Coconut2026.02 | 23.1 | 6 | |
| iCoTModel=LLaMA-3.2 Instruct 3B, Finetuning Setting=true2026.06 | 21.57 | 0 | |
| SoftCoT++Model=LLaMA-3.1 Instruct 8B, Finetuning Setting=true2026.06 | 20.86 | 120.76 | |
| LSTRCompression ratio=5, Ablation=w/o Sparse2026.02 | 20.7 | 5.8 | |
| CoconutBackbone=LLaMA-3.2-1B-Instruct2026.01 | 20.5 | 6 | |
| CoconutModel=LLaMA-3.2 Instruct 1B, Finetuning Setting=true2026.06 | 20.5 | 6 | |
| Assist-CoTModel=LLaMA-3.1 Instruct 8B, Finetuning Setting=true2026.06 | 19.93 | 121.04 | |
| iCoT2026.02 | 19.8 | 0 | |
| iCoT2026.02 | 19.8 | 0 | |
| iCoTBackbone=LLaMA-3.2-1B-Instruct2026.01 | 19.8 | 0 | |
| iCoTModel=LLaMA-3.2 Instruct 1B, Finetuning Setting=true2026.06 | 19.8 | 0 | |
| CoTModel=LLaMA-3.1 Instruct 8B, Finetuning Setting=true2026.06 | 19.17 | 121.3 | |
| iCoTModel=DeepSeek-R1-Distill-Qwen-1.5B, Finetuning Setting=true2026.06 | 18.87 | 0 | |
| SoftCoT++Model=LLaMA-3.2 Instruct 3B, Finetuning Setting=true2026.06 | 18.59 | 120.77 | |
| SoftCoT++Model=DeepSeek-R1-Distill-Qwen-1.5B, Finetuning Setting=true2026.06 | 18.19 | 120.64 | |
| Assist-CoTModel=LLaMA-3.2 Instruct 3B, Finetuning Setting=true2026.06 | 17.9 | 120.98 | |
| Assist-CoTModel=DeepSeek-R1-Distill-Qwen-1.5B, Finetuning Setting=true2026.06 | 17.36 | 120.81 | |
| CoTModel=LLaMA-3.2 Instruct 3B, Finetuning Setting=true2026.06 | 17.1 | 121.39 | |
| SoftCoT++Model=LLaMA-3.2 Instruct 1B, Finetuning Setting=true2026.06 | 17.04 | 119.73 | |
| RoT (Ours)Model=LLaVa-V1.6-Mistral-7B2026.01 | 16.3 | 32 | |
| Assist-CoTModel=LLaMA-3.2 Instruct 1B, Finetuning Setting=true2026.06 | 16.3 | 119.85 | |
| CoTModel=DeepSeek-R1-Distill-Qwen-1.5B, Finetuning Setting=true2026.06 | 15.94 | 121.57 | |
| CoTModel=LLaMA-3.2 Instruct 1B, Finetuning Setting=true2026.06 | 15.7 | 120.37 | |
| SFT-w/o CoTModel=Qwen3-VL-2B-Instruct2026.01 | 15.6 | 0 | |
| Distill2026.02 | 13.3 | 6 | |
| CODI2026.02 | 13.3 | 6 | |
| CODIBackbone=LLaMA-3.2-1B-Instruct2026.01 | 13.3 | 6 | |
| CODIModel=LLaMA-3.2 Instruct 1B, Finetuning Setting=true2026.06 | 13.3 | 6 | |
| SFT-w/o CoTModel=LLaVa-V1.6-Mistral-7B2026.01 | 10.8 | 0 |