Speculative Decoding on SpecBench and HumanEval
392.7Throughput (tokens/s)MicroSpec
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MicroSpecPruning Strategy=Variant by Context, Extra Training=None, Active Vocab. Size=<3k, Backbone=Llama-3-8B-Instruct2026.04 | 392.7 | 3.59 | |
| DynaSpecPruning Strategy=Route to Fixed Clusters, Extra Training=1 MLP, Active Vocab. Size=27k, Backbone=Llama-3-8B-Instruct2026.04 | 378.1 | 3.78 | |
| FR-SpecPruning Strategy=High-Frequency Tokens, Extra Training=None, Active Vocab. Size=32k, Backbone=Llama-3-8B-Instruct2026.04 | 369.7 | 3.62 | |
| CORALPruning Strategy=Route to Fixed Clusters, Extra Training=1-layer FFN, Active Vocab. Size=32k, Backbone=Llama-3-8B-Instruct2026.04 | 359.9 | 3.92 | |
| EAGLE-2Pruning Strategy=None (Full Vocabulary), Extra Training=None, Active Vocab. Size=128k, Backbone=Llama-3-8B-Instruct2026.04 | 336.9 | 3.8 |