Multi-path Speculative Decoding on Qwen (test)
22.54Throughput (tokens/s)SpecInfer NDE
Evaluation results
| Method | Links | |
|---|---|---|
| SpecInfer NDEExpansion Mode=Neural Dynamic Expansion2026.02 | 22.54 | |
| SpecTr NDEExpansion Mode=Neural Dynamic Expansion2026.02 | 21.74 | |
| NaiveTree NDEExpansion Mode=Neural Dynamic Expansion2026.02 | 21.67 | |
| TraversalExpansion Mode=Static2026.02 | 21.35 | |
| NSS NDEExpansion Mode=Neural Dynamic Expansion2026.02 | 20.29 | |
| Khisti NDEExpansion Mode=Neural Dynamic Expansion2026.02 | 18.41 |