Mathematical Reasoning on GSM-Hard OOD 1.0 (test)
12Greedy Success RateRiM
Evaluation results
| Method | Links | ||
|---|---|---|---|
| RiMModel=Llama-3.2-3B, Variant=Final block, TTFT (ms)=27.92026.05 | 12 | 14.1 | |
| RiMModel=Llama-3.2-1B, Variant=Final block, TTFT (ms)=16.12026.05 | 10.5 | 13.8 | |
| CoconutModel=Llama-3.2-3B, Variant=w/ Stage 0, TTFT (ms)=188.82026.05 | 10.2 | 13.5 | |
| CoconutModel=Llama-3.2-1B, Variant=w/ Stage 0, TTFT (ms)=108.32026.05 | 8.5 | 12.2 | |
| SFTModel=Llama-3.2-3B, Variant=w/o CoT, TTFT (ms)=27.92026.05 | 8.5 | 10.8 | |
| RiMModel=GPT-2, Variant=Final block, TTFT (ms)=7.62026.05 | 7.8 | 11.2 | |
| CoconutModel=GPT-2, Variant=w/ Stage 0, TTFT (ms)=53.42026.05 | 7.1 | 10.7 | |
| SFTModel=Llama-3.2-1B, Variant=w/o CoT, TTFT (ms)=16.12026.05 | 5.3 | 9.5 | |
| SFTModel=GPT-2, Variant=w/o CoT, TTFT (ms)=7.62026.05 | 3.5 | 7.6 |