Instruction Following on Alpaca (Speedup and Drafting Cost Analysis)
6.27Average Accepted LengthEAGLE-3
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| EAGLE-3Model=Llama-3.1-8B, Temperature=02026.05 | 6.27 | 3.35 | 31 | |
| EAGLE-3Model=Llama-3.1-8B, Temperature=1.02026.05 | 5.17 | 2.36 | 30 | |
| SpecBlock+OSDModel=Llama-3.1-8B, Temperature=02026.05 | 4.74 | 3.43 | 18 | |
| SpecBlock+adaptModel=Llama-3.1-8B, Temperature=02026.05 | 4.72 | 3.47 | 15 | |
| SpecBlockModel=Llama-3.1-8B, Temperature=02026.05 | 4.66 | 3.4 | 16 | |
| EAGLE-3Model=Qwen3-8B, Temperature=02026.05 | 4.29 | 2.35 | 29 | |
| CATS w/ EAGLEDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=tree base, Backbone=Vicuna-7B, Drafting steps=5, top-K=102026.05 | 4.2745 | 4.29 | — | |
| SpecBlock+OSDModel=Llama-3.1-8B, Temperature=1.02026.05 | 4.25 | 2.66 | 17 | |
| FalconModel=Llama-3.1-8B, Temperature=02026.05 | 4.23 | 2.86 | 26 | |
| SpecBlock+adaptModel=Llama-3.1-8B, Temperature=1.02026.05 | 4.22 | 2.72 | 15 | |
| EAGLE-3Model=Qwen3-8B, Temperature=1.02026.05 | 4.14 | 1.94 | 29 | |
| EAGLE-3Model=Qwen3-32B, Temperature=02026.05 | 4.09 | 2.28 | 24 | |
| SpecBlockModel=Llama-3.1-8B, Temperature=1.02026.05 | 4.02 | 2.57 | 14 | |
| EAGLE-3Model=Qwen3-32B, Temperature=1.02026.05 | 3.94 | 1.84 | 25 | |
| SpecBlock+OSDModel=Qwen3-8B, Temperature=02026.05 | 3.8 | 2.62 | 19 | |
| SpecBlock+adaptModel=Qwen3-8B, Temperature=02026.05 | 3.78 | 2.64 | 17 | |
| SpecBlockModel=Qwen3-8B, Temperature=02026.05 | 3.73 | 2.58 | 14 | |
| SpecBlock+OSDModel=Qwen3-8B, Temperature=1.02026.05 | 3.72 | 2.14 | 19 | |
| SpecBlock+adaptModel=Qwen3-8B, Temperature=1.02026.05 | 3.69 | 2.19 | 18 | |
| CATSDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.05 | 3.6742 | 3.82 | — | |
| FalconModel=Llama-3.1-8B, Temperature=1.02026.05 | 3.64 | 2.11 | 27 | |
| SpecBlock+OSDModel=Qwen3-32B, Temperature=02026.05 | 3.62 | 2.5 | 15 | |
| SpecBlock+adaptModel=Qwen3-32B, Temperature=02026.05 | 3.61 | 2.51 | 13 | |
| SpecBlockModel=Qwen3-8B, Temperature=1.02026.05 | 3.59 | 2.06 | 14 | |
| EAGLE3 (Qwen3.5 9B)Normalization=Post-norm draft, Decoding mode=thinking2026.05 | 3.57 | — | — | |
| SpecBlock+OSDModel=Qwen3-32B, Temperature=1.02026.05 | 3.55 | 2 | 14 | |
| SpecBlockModel=Qwen3-32B, Temperature=02026.05 | 3.54 | 2.48 | 11 | |
| EAGLE-selfDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=tree base, Backbone=Vicuna-7B, Drafting steps=5, top-K=102026.05 | 3.5334 | 2.68 | — | |
| SpecBlock+adaptModel=Qwen3-32B, Temperature=1.02026.05 | 3.53 | 2.06 | 13 | |
| EAGLE3 (Qwen3.5 9B)Normalization=Pre-norm draft, Decoding mode=thinking2026.05 | 3.49 | — | — | |
| CATS w/ EAGLEDecoding Strategy=Greedy, Decoding Structure=tree base, Backbone=Vicuna-7B, Drafting steps=5, top-K=102026.05 | 3.4339 | 3.45 | — | |
| SpecBlockModel=Qwen3-32B, Temperature=1.02026.05 | 3.42 | 1.94 | 11 | |
| ParallelSpecModel=Llama-3.1-8B, Temperature=02026.05 | 3.28 | 2.39 | 8 | |
| EAGLE-selfDecoding Strategy=Greedy, Decoding Structure=tree base, Backbone=Vicuna-7B, Drafting steps=5, top-K=102026.05 | 2.9306 | 2.21 | — | |
| EAGLE-selfDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.05 | 2.8644 | 2.17 | — | |
| ParallelSpecModel=Llama-3.1-8B, Temperature=1.02026.05 | 2.85 | 1.78 | 9 | |
| KangarooDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.05 | 2.8348 | 2.18 | — | |
| CATSDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.05 | 2.828 | 2.94 | — | |
| EAGLE3 (Qwen3.5 9B)Normalization=Post-norm draft, Decoding mode=no-thinking2026.05 | 2.79 | — | — | |
| EAGLE3 (Qwen3.5 9B)Normalization=Pre-norm draft, Decoding mode=no-thinking2026.05 | 2.73 | — | — | |
| SpSModel=Llama-3.1-8B, Temperature=02026.05 | 2.7 | 1.48 | 38 | |
| MedusaDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base, Backbone=Vicuna-7B, Number of heads=22026.05 | 2.6179 | 2.63 | — | |
| MedusaModel=Llama-3.1-8B, Temperature=02026.05 | 2.44 | 2.01 | 6 | |
| MedusaDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B, Number of heads=22026.05 | 2.3917 | 2.41 | — | |
| SpSModel=Llama-3.1-8B, Temperature=1.02026.05 | 2.33 | 1.06 | 40 | |
| KangarooDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.05 | 2.3131 | 1.79 | — | |
| EAGLE-selfDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.05 | 2.3104 | 1.75 | — | |
| MedusaModel=Llama-3.1-8B, Temperature=1.02026.05 | 2.07 | 1.5 | 7 | |
| LookaheadDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B2026.05 | 2.0623 | 2.08 | — | |
| RESTDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base, Backbone=Vicuna-7B2026.05 | 1.8594 | 1.9 | — | |
| RESTDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B2026.05 | 1.8406 | 1.88 | — |