Mathematical Reasoning on GSM8K ID 1.0 (test)
48.8GreedyRiM
Evaluation results
| Method | Links | ||
|---|---|---|---|
| RiMModel=Llama-3.2-3B, Variant=Final block, TTFT (ms)=27.92026.05 | 48.8 | 58.8 | |
| RiMModel=Llama-3.2-1B, Variant=Final block, TTFT (ms)=16.12026.05 | 42.1 | 56.1 | |
| CoconutModel=Llama-3.2-3B, Variant=w/ Stage 0, TTFT (ms)=188.82026.05 | 41.3 | 55.5 | |
| CoconutModel=Llama-3.2-1B, Variant=w/ Stage 0, TTFT (ms)=108.32026.05 | 36.9 | 51.1 | |
| SFTModel=Llama-3.2-3B, Variant=w/o CoT, TTFT (ms)=27.92026.05 | 36.2 | 45.9 | |
| RiMModel=GPT-2, Variant=Final block, TTFT (ms)=7.62026.05 | 33.6 | 49.1 | |
| CoconutModel=GPT-2, Variant=w/ Stage 0, TTFT (ms)=53.42026.05 | 31.1 | 45 | |
| SFTModel=Llama-3.2-1B, Variant=w/o CoT, TTFT (ms)=16.12026.05 | 23.9 | 41.7 | |
| SFTModel=GPT-2, Variant=w/o CoT, TTFT (ms)=7.62026.05 | 15.4 | 33.3 |