Multi-path Speculative Decoding on Gemma (test)
13.26Throughput (tokens/s)SpecInfer NDE
Evaluation results
| Method | Links | |
|---|---|---|
| SpecInfer NDEExpansion Mode=Neural Dynamic Expansion2026.02 | 13.26 | |
| SpecTr NDEExpansion Mode=Neural Dynamic Expansion2026.02 | 13.22 | |
| NaiveTree NDEExpansion Mode=Neural Dynamic Expansion2026.02 | 12.86 | |
| Khisti NDEExpansion Mode=Neural Dynamic Expansion2026.02 | 12.84 | |
| NSS NDEExpansion Mode=Neural Dynamic Expansion2026.02 | 12.4 | |
| TraversalExpansion Mode=Static2026.02 | 11.56 |