List your company
ResearchFormerly WizWand

Mathematical Reasoning on GSM-Hard OOD

19Greedy AccuracySFT
2.887.06511.2515.435May 28, 2026
Updated 5mo ago

Evaluation results

MethodLinks
SFTModel=Llama-3.2-3B, Variant=w/ CoT, TTFT (ms)=754.42026.0519———24.4
RiMModel=Llama-3.2-3B, Variant=Any block, TTFT (ms)=27.92026.0513.8———18.2
RiMModel=Llama-3.2-1B, Variant=Any block, TTFT (ms)=16.12026.0513———19.6
RiMBackbone=Llama-3.2-3B2026.0512.1————
RiMModel=Llama-3.2-3B, Variant=Final block, TTFT (ms)=27.92026.0512———14.1
SFTModel=Llama-3.2-1B, Variant=w/ CoT, TTFT (ms)=420.32026.0511.2———15.3
CoconutBackbone=Llama-3.2-1B2026.0511.2————
RiMBackbone=Llama-3.2-1B2026.0511.2————
DARTBackbone=Llama-3.2-1B2026.0510.9————
RiMModel=Llama-3.2-1B, Variant=Final block, TTFT (ms)=16.12026.0510.5———13.8
CoconutModel=Llama-3.2-3B, Variant=w/ Stage 0, TTFT (ms)=188.82026.0510.2———13.5
CoconutModel=Llama-3.2-3B, Variant=w/o Stage 0, TTFT (ms)=188.82026.0510.1———13.3
RiMModel=GPT-2, Variant=Any block, TTFT (ms)=7.62026.059.4———19
CoconutModel=Llama-3.2-1B, Variant=w/ Stage 0, TTFT (ms)=108.32026.058.5———12.2
SFTModel=Llama-3.2-3B, Variant=w/o CoT, TTFT (ms)=27.92026.058.5———10.8
SFTModel=GPT-2, Variant=w/ CoT, TTFT (ms)=213.72026.058.4———12.9
RiMBackbone=GPT-22026.058.4————
RiMModel=GPT-2, Variant=Final block, TTFT (ms)=7.62026.057.8———11.2
CoconutModel=GPT-2, Variant=w/ Stage 0, TTFT (ms)=53.42026.057.1———10.7
CoconutModel=Llama-3.2-1B, Variant=w/o Stage 0, TTFT (ms)=108.32026.056.8———11
SFTModel=Llama-3.2-1B, Variant=w/o CoT, TTFT (ms)=16.12026.055.3———9.5
CoconutModel=GPT-2, Variant=w/o Stage 0, TTFT (ms)=53.42026.055.2———8.9
SFTModel=GPT-2, Variant=w/o CoT, TTFT (ms)=7.62026.053.5———7.6
THOUGHTSTEERArchitecture=SimCoT (Llama-3.2) 1B, Decoding method=CODI2026.04—9.59.5100—
THOUGHTSTEERArchitecture=SimCoT (Llama-3.2) 3B, Decoding method=CODI2026.04—11.411.599.3—