Mathematical Reasoning on GSM8K (test) (Acc, Tok, Lat)
95.3Accuracy (GSM8K)Think
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ThinkBase Model=Qwen3-4B-Thinking-25072026.02 | 95.3 | 1,414 | 155.6 | |
| NoThink*Base Model=Qwen3-4B-Thinking-25072026.02 | 95.2 | 1,631 | 204.2 | |
| EAGLE3Base Model=Qwen3-4B-Thinking-25072026.02 | 94.8 | 1,408 | 140.3 | |
| DEERBase Model=Qwen3-4B-Thinking-25072026.02 | 94.3 | 960 | 230.3 | |
| SpecExitBase Model=Qwen3-4B-Thinking-25072026.02 | 93.8 | 649 | 75.8 | |
| EAGLE3Base Model=DeepSeek-R1-Distill-Llama-8B2026.02 | 79.3 | 976 | 276.9 | |
| VanillaBase Model=DeepSeek-R1-Distill-Llama-8B2026.02 | 76.4 | 1,008 | 629.4 | |
| SpecExitBase Model=DeepSeek-R1-Distill-Llama-8B2026.02 | 75.3 | 333 | 112.6 | |
| DEERBase Model=DeepSeek-R1-Distill-Llama-8B2026.02 | 74.7 | 710 | 484.8 | |
| NoThinkBase Model=DeepSeek-R1-Distill-Llama-8B2026.02 | 54.6 | 233 | 22.2 |