Multi-path Speculative Decoding on Average across models (Qwen, Gemma, Llama) (test)
16.69Throughput (tokens/s)SpecInfer NDE
Evaluation results
| Method | Links | |
|---|---|---|
| SpecInfer NDEExpansion Mode=Neural Dynamic Expansion2026.02 | 16.69 | |
| SpecTr NDEExpansion Mode=Neural Dynamic Expansion2026.02 | 16.4 | |
| NaiveTree NDEExpansion Mode=Neural Dynamic Expansion2026.02 | 16.18 | |
| TraversalExpansion Mode=Static2026.02 | 15.89 | |
| NSS NDEExpansion Mode=Neural Dynamic Expansion2026.02 | 15.05 | |
| Khisti NDEExpansion Mode=Neural Dynamic Expansion2026.02 | 14.83 |