Mathematical Reasoning on GSM-Hard OOD
19Greedy AccuracySFT
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| SFTModel=Llama-3.2-3B, Variant=w/ CoT, TTFT (ms)=754.42026.05 | 19 | — | — | — | 24.4 | |
| RiMModel=Llama-3.2-3B, Variant=Any block, TTFT (ms)=27.92026.05 | 13.8 | — | — | — | 18.2 | |
| RiMModel=Llama-3.2-1B, Variant=Any block, TTFT (ms)=16.12026.05 | 13 | — | — | — | 19.6 | |
| RiMBackbone=Llama-3.2-3B2026.05 | 12.1 | — | — | — | — | |
| RiMModel=Llama-3.2-3B, Variant=Final block, TTFT (ms)=27.92026.05 | 12 | — | — | — | 14.1 | |
| SFTModel=Llama-3.2-1B, Variant=w/ CoT, TTFT (ms)=420.32026.05 | 11.2 | — | — | — | 15.3 | |
| CoconutBackbone=Llama-3.2-1B2026.05 | 11.2 | — | — | — | — | |
| RiMBackbone=Llama-3.2-1B2026.05 | 11.2 | — | — | — | — | |
| DARTBackbone=Llama-3.2-1B2026.05 | 10.9 | — | — | — | — | |
| RiMModel=Llama-3.2-1B, Variant=Final block, TTFT (ms)=16.12026.05 | 10.5 | — | — | — | 13.8 | |
| CoconutModel=Llama-3.2-3B, Variant=w/ Stage 0, TTFT (ms)=188.82026.05 | 10.2 | — | — | — | 13.5 | |
| CoconutModel=Llama-3.2-3B, Variant=w/o Stage 0, TTFT (ms)=188.82026.05 | 10.1 | — | — | — | 13.3 | |
| RiMModel=GPT-2, Variant=Any block, TTFT (ms)=7.62026.05 | 9.4 | — | — | — | 19 | |
| CoconutModel=Llama-3.2-1B, Variant=w/ Stage 0, TTFT (ms)=108.32026.05 | 8.5 | — | — | — | 12.2 | |
| SFTModel=Llama-3.2-3B, Variant=w/o CoT, TTFT (ms)=27.92026.05 | 8.5 | — | — | — | 10.8 | |
| SFTModel=GPT-2, Variant=w/ CoT, TTFT (ms)=213.72026.05 | 8.4 | — | — | — | 12.9 | |
| RiMBackbone=GPT-22026.05 | 8.4 | — | — | — | — | |
| RiMModel=GPT-2, Variant=Final block, TTFT (ms)=7.62026.05 | 7.8 | — | — | — | 11.2 | |
| CoconutModel=GPT-2, Variant=w/ Stage 0, TTFT (ms)=53.42026.05 | 7.1 | — | — | — | 10.7 | |
| CoconutModel=Llama-3.2-1B, Variant=w/o Stage 0, TTFT (ms)=108.32026.05 | 6.8 | — | — | — | 11 | |
| SFTModel=Llama-3.2-1B, Variant=w/o CoT, TTFT (ms)=16.12026.05 | 5.3 | — | — | — | 9.5 | |
| CoconutModel=GPT-2, Variant=w/o Stage 0, TTFT (ms)=53.42026.05 | 5.2 | — | — | — | 8.9 | |
| SFTModel=GPT-2, Variant=w/o CoT, TTFT (ms)=7.62026.05 | 3.5 | — | — | — | 7.6 | |
| THOUGHTSTEERArchitecture=SimCoT (Llama-3.2) 1B, Decoding method=CODI2026.04 | — | 9.5 | 9.5 | 100 | — | |
| THOUGHTSTEERArchitecture=SimCoT (Llama-3.2) 3B, Decoding method=CODI2026.04 | — | 11.4 | 11.5 | 99.3 | — |