LLM Inference on GSM8K (val)
112Throughput (tok/s)ENTMTPτ
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ENTMTPτTree Configuration=per-step switching between conservative and aggressive trees2026.06 | 112 | 3.13 | 3.02 | |
| ENTMTP*Tree Configuration=per-task throughput-optimal tree2026.06 | 109.7 | 3.07 | 3.08 | |
| Hydra (default)Tree Configuration=authors’ published default trees2026.06 | 102.4 | 2.87 | 2.86 | |
| Medusa (default)Tree Configuration=authors’ published default trees2026.06 | 87.4 | 2.59 | 2.51 | |
| Vanilla VicunaBase Model=Vicuna-7B v1.32026.06 | 33.7 | 1 | 1 |