Speculative Decoding on LWM 16K context
1.94SpeedupBudgetDraft
Evaluation results
| Method | Links | |
|---|---|---|
| BudgetDraftDrafter=68M, KV budget (B)=256, Draft length (γ)=52026.05 | 1.94 | |
| BudgetDraftDrafter=68M, KV budget (B)=512, Draft length (γ)=52026.05 | 1.89 | |
| BudgetDraftDrafter=68M, KV budget (B)=1024, Draft length (γ)=52026.05 | 1.77 | |
| BudgetDraftDrafter=68M, KV budget (B)=2048, Draft length (γ)=52026.05 | 1.52 | |
| EAGLE-3Drafter=draft head, Draft length (γ)=52026.05 | 1.36 | |
| TriForceDrafter=68M+7B, Draft length (γ)=52026.05 | 1.19 | |
| ARDrafter=–, Draft length (γ)=52026.05 | 1 | |
| SD (sparse/full)Drafter=68M, Draft length (γ)=52026.05 | 0.78 |