Multi-path speculative decoding on Qwen held-out (test)
1.31Throughput Ratio ImprovementNaiveTree NDE
Evaluation results
| Method | Links | |
|---|---|---|
| NaiveTree NDEVerification algorithm=NaiveTree, Neural Dynamic Expansion=true2026.02 | 1.31 | |
| NSS NDEVerification algorithm=NSS, Neural Dynamic Expansion=true2026.02 | 1.2 | |
| SpecInfer NDEVerification algorithm=SpecInfer, Neural Dynamic Expansion=true2026.02 | 1.15 | |
| SpecTr NDEVerification algorithm=SpecTr, Neural Dynamic Expansion=true2026.02 | 1.14 | |
| Khisti NDEVerification algorithm=Khisti, Neural Dynamic Expansion=true2026.02 | 0.99 |