Multi-path speculative decoding on Llama (held-out test)
1.21Throughput Ratio ImprovementNaiveTree NDE
Evaluation results
| Method | Links | |
|---|---|---|
| NaiveTree NDEVerification algorithm=NaiveTree, Neural Dynamic Expansion=true2026.02 | 1.21 | |
| Khisti NDEVerification algorithm=Khisti, Neural Dynamic Expansion=true2026.02 | 0.97 | |
| NSS NDEVerification algorithm=NSS, Neural Dynamic Expansion=true2026.02 | 0.96 | |
| SpecInfer NDEVerification algorithm=SpecInfer, Neural Dynamic Expansion=true2026.02 | 0.96 | |
| SpecTr NDEVerification algorithm=SpecTr, Neural Dynamic Expansion=true2026.02 | 0.96 |