Throughput on QA (Question Answering)
3.66Speedup FactorEagle3+LTD
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Eagle3+LTDModel=Vicuna-13B-v1.3, Temperature=0, Decoding Mode=Greedy2026.03 | 3.66 | — | — | 5.37 | — | |
| Eagle3+LTDModel=DeepSeek-R1-Distill-LLaMA 8B, Temperature=0, Decoding Mode=Greedy2026.03 | 3.62 | — | — | 5.37 | — | |
| Eagle3+LTDModel=Llama-3.1-8B-Instruct, Temperature=0, Decoding Mode=Greedy2026.03 | 3.41 | — | — | 5.44 | — | |
| TALONModel=Llama3-8B, Temperature=02026.01 | 3.27 | — | 5.09 | — | — | |
| TALONModel=Vicuna-13B, Temperature=02026.01 | 3.26 | — | 5.03 | — | — | |
| Eagle3+LTDModel=Vicuna-13B-v1.3, Temperature=12026.03 | 3.26 | — | — | 4.97 | — | |
| TALONModel=DSL-8B, Temperature=02026.01 | 3.12 | — | 4.66 | — | — | |
| OPT-TreeModel=Vicuna-13B, Temperature=02026.01 | 3.1 | — | 5.22 | — | — | |
| Eagle3Model=Vicuna-13B-v1.3, Temperature=12026.03 | 3.08 | — | — | 4.64 | — | |
| EAGLE-3Model=Vicuna-13B, Temperature=02026.01 | 3.03 | — | 5.24 | — | — | |
| Eagle3+Spec++Model=Vicuna-13B-v1.3, Temperature=12026.03 | 3.02 | — | — | 4.55 | — | |
| EAGLE-3Model=Llama3-8B, Temperature=02026.01 | 3.01 | — | 5.36 | — | — | |
| Eagle3+GSModel=Vicuna-13B-v1.3, Temperature=12026.03 | 3 | — | — | 4.93 | — | |
| Eagle3+GTModel=Vicuna-13B-v1.3, Temperature=12026.03 | 2.99 | — | — | 4.44 | — | |
| Eagle3+DDDModel=Vicuna-13B-v1.3, Temperature=12026.03 | 2.96 | — | — | 4.11 | — | |
| Eagle3+DiscoModel=Vicuna-13B-v1.3, Temperature=12026.03 | 2.88 | — | — | 4.59 | — | |
| EAGLE-3Model=DSL-8B, Temperature=02026.01 | 2.86 | — | 5.02 | — | — | |
| Eagle3+C2TModel=Vicuna-13B-v1.3, Temperature=12026.03 | 2.83 | — | — | 4.65 | — | |
| Eagle3+SvipModel=Vicuna-13B-v1.3, Temperature=12026.03 | 2.61 | — | — | 4.44 | — | |
| Eagle3+LTDModel=DeepSeek-R1-Distill-LLaMA 8B, Temperature=12026.03 | 2.57 | — | — | 4.39 | — | |
| TALONModel=Qwen3-8B, Temperature=02026.01 | 2.46 | — | 3.41 | — | — | |
| Eagle3+GTModel=Llama-3.1-8B-Instruct, Temperature=12026.03 | 2.46 | — | — | 3.24 | — | |
| Eagle3Model=DeepSeek-R1-Distill-LLaMA 8B, Temperature=12026.03 | 2.44 | — | — | 4.06 | — | |
| Eagle3+LTDModel=Llama-3.1-8B-Instruct, Temperature=12026.03 | 2.39 | — | — | 3.26 | — | |
| Eagle3+DDDModel=Llama-3.1-8B-Instruct, Temperature=12026.03 | 2.36 | — | — | 2.95 | — | |
| Eagle3+Spec++Model=Llama-3.1-8B-Instruct, Temperature=12026.03 | 2.35 | — | — | 3.28 | — | |
| Eagle3+GSModel=DeepSeek-R1-Distill-LLaMA 8B, Temperature=12026.03 | 2.33 | — | — | 4.44 | — | |
| Eagle3+SvipModel=Llama-3.1-8B-Instruct, Temperature=12026.03 | 2.31 | — | — | 3.2 | — | |
| Eagle3+GSModel=Llama-3.1-8B-Instruct, Temperature=12026.03 | 2.31 | — | — | 3.4 | — | |
| Eagle3Model=Llama-3.1-8B-Instruct, Temperature=12026.03 | 2.27 | — | — | 3.06 | — | |
| Eagle3+C2TModel=Llama-3.1-8B-Instruct, Temperature=12026.03 | 2.27 | — | — | 3.06 | — | |
| Eagle3+DiscoModel=Llama-3.1-8B-Instruct, Temperature=12026.03 | 2.2 | — | — | 2.88 | — | |
| EAGLE-3Model=Qwen3-8B, Temperature=02026.01 | 2.09 | — | 3.46 | — | — | |
| Eagle3+LTDModel=Qwen3-32B, Temperature=0, Decoding Mode=Greedy2026.03 | 2.06 | — | — | 2.6 | — | |
| EAGLE-2(+FR 32k)Backbone=Llama-3-8B, Temperature=1, Implementation=Our Implementation, Feature=FR 32k2025.02 | 2.02 | 182.59 | — | — | — | |
| TALONModel=Qwen3-32B, Temperature=02026.01 | 1.99 | — | 2.64 | — | — | |
| Eagle3+LTDModel=Qwen3-14B, Temperature=0, Decoding Mode=Greedy2026.03 | 1.98 | — | — | 2.62 | — | |
| HYDRAModel=Vicuna-13B, Temperature=02026.01 | 1.95 | — | 2.88 | — | — | |
| Eagle3+LTDModel=Qwen3 32B, Temperature=12026.03 | 1.88 | — | — | 2.41 | — | |
| Eagle3+LTDModel=Qwen3 14B, Temperature=12026.03 | 1.82 | — | — | 2.55 | — | |
| EAGLE-3Model=Qwen3-32B, Temperature=02026.01 | 1.78 | — | 2.66 | — | — | |
| EAGLE-2Backbone=Llama-3-8B, Temperature=1, Implementation=Our Implementation2025.02 | 1.71 | 155.1 | — | — | — | |
| Eagle3+GSModel=Qwen3 14B, Temperature=12026.03 | 1.68 | — | — | 2.57 | — | |
| MEDUSAModel=Vicuna-13B, Temperature=02026.01 | 1.62 | — | 2.1 | — | — | |
| Eagle3+GSModel=Qwen3 32B, Temperature=12026.03 | 1.33 | — | — | 2.46 | — | |
| SDModel=Vicuna-13B, Temperature=02026.01 | 1.2 | — | 1.97 | — | — | |
| VanillaBackbone=Llama-3-8B, Temperature=1, Implementation=Our Implementation2025.02 | 1 | 90.58 | — | — | — | |
| Accelerate2026.02 | — | 1.16 | — | — | 138.42 | |
| DDDTemperature=0, Backbone=Ling-flash-2.0, Batch Size=1, TP (Tensor Parallelism)=4, steps=4, topk=8, draft_tokens=322026.05 | — | 187.23 | — | — | — | |
| DDDTemperature=1, Backbone=Ling-flash-2.0, Batch Size=1, TP (Tensor Parallelism)=4, steps=4, topk=8, draft_tokens=322026.05 | — | 180.36 | — | — | — | |
| EAGLE-3Temperature=0, Backbone=Ling-flash-2.0, Batch Size=1, TP (Tensor Parallelism)=4, steps=4, topk=8, draft_tokens=322026.05 | — | 198.58 | — | — | — | |
| EAGLE-3Temperature=1, Backbone=Ling-flash-2.0, Batch Size=1, TP (Tensor Parallelism)=4, steps=4, topk=8, draft_tokens=322026.05 | — | 188.61 | — | — | — | |
| EVICTTemperature=0, Backbone=Ling-flash-2.0, Batch Size=1, TP (Tensor Parallelism)=4, steps=4, topk=8, draft_tokens=322026.05 | — | 225.28 | — | — | — | |
| EVICTTemperature=1, Backbone=Ling-flash-2.0, Batch Size=1, TP (Tensor Parallelism)=4, steps=4, topk=8, draft_tokens=322026.05 | — | 208.96 | — | — | — | |
| Fate2026.02 | — | 7.83 | — | — | 24.23 | |
| HybriMoE2026.02 | — | 11.83 | — | — | 15.45 | |
| llama.cppspeculative decoding=false2026.02 | — | 15.09 | — | — | 11.07 | |
| llama.cppspeculative decoding=true2026.02 | — | 16.49 | — | — | 10.43 | |
| LookaheadTemperature=0, Backbone=Ling-flash-2.0, Batch Size=1, TP (Tensor Parallelism)=42026.05 | — | 121.47 | — | — | — | |
| LookaheadTemperature=1, Backbone=Ling-flash-2.0, Batch Size=1, TP (Tensor Parallelism)=42026.05 | — | 117.18 | — | — | — | |
| Mixtral Offload2026.02 | — | 2.33 | — | — | 70.86 | |
| MoE-Infinity2026.02 | — | 3.44 | — | — | 46.13 | |
| MoE-SpAc2026.02 | — | 23.56 | — | — | 8.36 | |
| SP-MoE2026.02 | — | 4.55 | — | — | 34.62 | |
| VanillaTemperature=0, Backbone=Ling-flash-2.0, Batch Size=1, TP (Tensor Parallelism)=42026.05 | — | 175.32 | — | — | — | |
| VanillaTemperature=1, Backbone=Ling-flash-2.0, Batch Size=1, TP (Tensor Parallelism)=42026.05 | — | 170.78 | — | — | — | |
| vLLM2026.02 | — | 2.68 | — | — | 57.43 |