LLM Inference on HumanEval (val)
124.7Throughput (tok/s)ENTMTPτ
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ENTMTPτTree Configuration=per-step switching between conservative and aggressive trees2026.06 | 124.7 | 3.26 | 3.2 | |
| ENTMTP*Tree Configuration=per-task throughput-optimal tree2026.06 | 123.4 | 3.21 | 3.28 | |
| Hydra (default)Tree Configuration=authors’ published default trees2026.06 | 109 | 2.85 | 3.06 | |
| Medusa (default)Tree Configuration=authors’ published default trees2026.06 | 91.2 | 2.38 | 2.87 | |
| Vanilla VicunaBase Model=Vicuna-7B v1.32026.06 | 38.2 | 1 | 1 |