Speculative Decoding on LWM 8K context
2.54SpeedupBudgetDraft
Evaluation results
| Method | Links | |
|---|---|---|
| BudgetDraftDrafter=68M, KV budget (B)=256, Draft length (γ)=52026.05 | 2.54 | |
| BudgetDraftDrafter=68M, KV budget (B)=512, Draft length (γ)=52026.05 | 2.54 | |
| BudgetDraftDrafter=68M, KV budget (B)=1024, Draft length (γ)=52026.05 | 2.54 | |
| BudgetDraftDrafter=68M, KV budget (B)=2048, Draft length (γ)=52026.05 | 2.54 | |
| EAGLE-3Drafter=draft head, Draft length (γ)=52026.05 | 1.64 | |
| TriForceDrafter=68M+7B, Draft length (γ)=52026.05 | 1.21 | |
| SD (sparse/full)Drafter=68M, Draft length (γ)=52026.05 | 1.19 | |
| ARDrafter=–, Draft length (γ)=52026.05 | 1 |