Speculative Decoding on GGUF Prompt Set
3.78TPSspecdec_plus_approx*
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| specdec_plus_approx*repeat_count=15, hardware=GGUF, prompt_set=identical2026.07 | 3.78 | 0.771 | 15.3 | 0.758 | 0.113 | 0.001 | |
| fixed_depth (static, ref.)repeat_count=15, hardware=GGUF, prompt_set=identical2026.07 | 3.25 | 0.664 | 10.8 | 0.747 | 0.163 | — | |
| ensemble (AdaptiveSD)repeat_count=15, hardware=GGUF, prompt_set=identical2026.07 | 3.2 | 0.653 | 10.3 | 0.642 | 0.212 | 0.46 | |
| parallel_sdrepeat_count=15, hardware=GGUF, prompt_set=identical2026.07 | 3.19 | 0.651 | 9.6 | 0.753 | 0.123 | 0.425 | |
| bandit (AdaptiveSD)repeat_count=15, hardware=GGUF, prompt_set=identical2026.07 | 3.15 | 0.642 | 9.5 | 0.696 | 0.195 | 0.453 | |
| dynamic_lookahead*repeat_count=15, hardware=GGUF, prompt_set=identical2026.07 | 3.11 | 0.634 | 8.9 | 0.751 | 0.207 | 0.155 | |
| heuristic (AdaptiveSD)repeat_count=15, hardware=GGUF, prompt_set=identical2026.07 | 3 | 0.613 | 8.8 | 0.819 | 0.188 | 0.044 | |
| ema (AdaptiveSD)repeat_count=15, hardware=GGUF, prompt_set=identical2026.07 | 2.62 | 0.535 | 8 | 0.933 | 0.27 | 0.001 |