Multi-turn Conversation on MT-Bench (Speedup and Average Acceptance Length)
4.64SpeedupEagle3+LTD
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Eagle3+LTDModel=Vicuna-13B-v1.3, Temperature=0, Decoding Mode=Greedy2026.03 | 4.64 | 8.24 | |
| Eagle3+GSModel=Vicuna-13B-v1.3, Temperature=0, Decoding Mode=Greedy2026.03 | 4.53 | 7.99 | |
| Eagle3Model=Vicuna-13B-v1.3, Temperature=0, Decoding Mode=Greedy2026.03 | 4.48 | 7.22 | |
| Eagle3+DiscoModel=Vicuna-13B-v1.3, Temperature=0, Decoding Mode=Greedy2026.03 | 4.48 | 7.08 | |
| Eagle3+C2TModel=Vicuna-13B-v1.3, Temperature=0, Decoding Mode=Greedy2026.03 | 4.33 | 7.22 | |
| Eagle3+Spec++Model=Vicuna-13B-v1.3, Temperature=0, Decoding Mode=Greedy2026.03 | 4.25 | 6.78 | |
| Eagle3+DDDModel=Vicuna-13B-v1.3, Temperature=0, Decoding Mode=Greedy2026.03 | 4.15 | 6.69 | |
| Eagle3+GTModel=Vicuna-13B-v1.3, Temperature=0, Decoding Mode=Greedy2026.03 | 4.15 | 6.85 | |
| Eagle3+LTDModel=DeepSeek-R1-Distill-LLaMA 8B, Temperature=0, Decoding Mode=Greedy2026.03 | 4.05 | 6.32 | |
| Eagle3+SvipModel=Vicuna-13B-v1.3, Temperature=0, Decoding Mode=Greedy2026.03 | 4.02 | 7.74 | |
| Eagle3+LTDModel=Llama-3.1-8B-Instruct, Temperature=0, Decoding Mode=Greedy2026.03 | 3.96 | 6.74 | |
| Eagle3+GSModel=Llama-3.1-8B-Instruct, Temperature=0, Decoding Mode=Greedy2026.03 | 3.83 | 6.75 | |
| Eagle3+Spec++Model=Llama-3.1-8B-Instruct, Temperature=0, Decoding Mode=Greedy2026.03 | 3.82 | 6.09 | |
| Eagle3+C2TModel=Llama-3.1-8B-Instruct, Temperature=0, Decoding Mode=Greedy2026.03 | 3.8 | 6.39 | |
| Eagle3+GTModel=Llama-3.1-8B-Instruct, Temperature=0, Decoding Mode=Greedy2026.03 | 3.79 | 5.86 | |
| Eagle3Model=Llama-3.1-8B-Instruct, Temperature=0, Decoding Mode=Greedy2026.03 | 3.7 | 6.39 | |
| Eagle3+DiscoModel=Llama-3.1-8B-Instruct, Temperature=0, Decoding Mode=Greedy2026.03 | 3.7 | 6.11 | |
| Eagle3+GSModel=DeepSeek-R1-Distill-LLaMA 8B, Temperature=0, Decoding Mode=Greedy2026.03 | 3.69 | 6.42 | |
| Eagle3+DDDModel=Llama-3.1-8B-Instruct, Temperature=0, Decoding Mode=Greedy2026.03 | 3.67 | 5.41 | |
| Eagle3Model=DeepSeek-R1-Distill-LLaMA 8B, Temperature=0, Decoding Mode=Greedy2026.03 | 3.67 | 5.82 | |
| Eagle3+SvipModel=Llama-3.1-8B-Instruct, Temperature=0, Decoding Mode=Greedy2026.03 | 3.65 | 6.21 | |
| PARD-2Target Model=Q3 14B, Temperature=0, Inference draft length=162026.05 | 3.37 | 3.96 | |
| PARD-2Target Model=L3.1 8B, Temperature=0, Inference draft length=162026.05 | 3.25 | 4.22 | |
| PARD-2Target Model=Q3 8B, Temperature=0, Inference draft length=162026.05 | 3.16 | 3.89 | |
| PARDTarget Model=L3.1 8B, Temperature=0, Inference draft length=162026.05 | 2.85 | 3.57 | |
| PARDTarget Model=Qwen3-14B, Inference Draft Length=16, Temperature=02026.05 | 2.78 | 3.26 | |
| PARDTarget Model=Q3 14B, Temperature=0, Inference draft length=162026.05 | 2.78 | 3.26 | |
| PARD-2Target Model=Qwen3-14B, Inference Draft Length=16, Temperature=02026.05 | 2.76 | 3.37 | |
| PARD-2Target Model=Qwen3-8B, Inference Draft Length=16, Temperature=02026.05 | 2.74 | 3.39 | |
| PARDTarget Model=Qwen3-8B, Inference Draft Length=16, Temperature=02026.05 | 2.68 | 3.79 | |
| PARDTarget Model=Q3 8B, Temperature=0, Inference draft length=162026.05 | 2.68 | 3.79 | |
| PARD-2Target Model=Qwen3-32B, Inference Draft Length=16, Temperature=02026.05 | 2.67 | 3.15 | |
| PARDTarget Model=Qwen3-32B, Inference Draft Length=16, Temperature=02026.05 | 2.64 | 3.05 | |
| SlimSpecTarget Model=Llama-3.1-8B-Instruct, Batch size=1, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 2.6 | — | |
| SpecVocabTarget Model=Llama-3.1-8B-Instruct, Batch size=1, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 2.59 | — | |
| FlexDraftModel=Qwen3-8B, Scaled training=true2026.05 | 2.58 | 4.16 | |
| DFlashTarget Model=Q3 8B, Temperature=0, Inference draft length=162026.05 | 2.57 | 3.1 | |
| EAGLE-3Target Model=L3.1 8B, Temperature=0, Inference draft length=82026.05 | 2.54 | 3.6 | |
| dFlashModel=Qwen3-8B, Scaled training=true2026.05 | 2.53 | 4.29 | |
| VocabTrim-TTarget Model=Llama-3.1-8B-Instruct, Batch size=1, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 2.47 | — | |
| Eagle3+LTDModel=Qwen3-14B, Temperature=0, Decoding Mode=Greedy2026.03 | 2.44 | 3.34 | |
| Eagle3+LTDModel=Qwen3-32B, Temperature=0, Decoding Mode=Greedy2026.03 | 2.42 | 3.17 | |
| Eagle3+GSModel=Qwen3-14B, Temperature=0, Decoding Mode=Greedy2026.03 | 2.34 | 3.4 | |
| DARTModel=Qwen3-8B, Scaled training=true2026.05 | 2.27 | 3.03 | |
| Full VocabTarget Model=Llama-3.1-8B-Instruct, Batch size=1, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 2.23 | — | |
| EAGLE-3Model=Qwen3-8B, Scaled training=true2026.05 | 1.89 | 3.04 | |
| Eagle3+GSModel=Qwen3-32B, Temperature=0, Decoding Mode=Greedy2026.03 | 1.76 | 3.3 | |
| SlimSpecTarget Model=GPT-OSS-20B, Batch size=1, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.76 | — | |
| SpecVocabTarget Model=GPT-OSS-20B, Batch size=1, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.74 | — | |
| EAGLE-3Target Model=Q3 14B, Temperature=0, Inference draft length=82026.05 | 1.72 | 1.97 | |
| VocabTrim-TTarget Model=GPT-OSS-20B, Batch size=1, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.72 | — | |
| EAGLE-3Target Model=Q3 8B, Temperature=0, Inference draft length=82026.05 | 1.71 | 2.05 | |
| VocabTrim-TTarget Model=Qwen3-30B-A3B-Instruct, Batch size=1, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.66 | — | |
| SpecVocabTarget Model=Qwen3-30B-A3B-Instruct, Batch size=1, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.66 | — | |
| SlimSpecTarget Model=Qwen3-30B-A3B-Instruct, Batch size=1, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.66 | — | |
| Apple MTPModel=Qwen3-8B, Scaled training=true2026.05 | 1.63 | 2.11 | |
| Full VocabTarget Model=Qwen3-30B-A3B-Instruct, Batch size=1, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.55 | — | |
| SlimSpecTarget Model=GPT-OSS-20B, Batch size=64, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.52 | — | |
| VocabTrim-TTarget Model=GPT-OSS-20B, Batch size=64, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.5 | — | |
| SlimSpecTarget Model=Llama-3.1-8B-Instruct, Batch size=64, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.43 | — | |
| Full VocabTarget Model=GPT-OSS-20B, Batch size=1, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.43 | — | |
| SpecVocabTarget Model=GPT-OSS-20B, Batch size=64, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.43 | — | |
| SpecVocabTarget Model=Llama-3.1-8B-Instruct, Batch size=64, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.36 | — | |
| Full VocabTarget Model=GPT-OSS-20B, Batch size=64, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.36 | — | |
| Full VocabTarget Model=Llama-3.1-8B-Instruct, Batch size=64, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.35 | — | |
| VocabTrim-TTarget Model=Qwen3-30B-A3B-Instruct, Batch size=64, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.31 | — | |
| SlimSpecTarget Model=Qwen3-30B-A3B-Instruct, Batch size=64, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.31 | — | |
| VocabTrim-TTarget Model=Llama-3.1-8B-Instruct, Batch size=64, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.29 | — | |
| SpecVocabTarget Model=Qwen3-30B-A3B-Instruct, Batch size=64, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.28 | — | |
| Full VocabTarget Model=Qwen3-30B-A3B-Instruct, Batch size=64, Temperature=0, Inference Framework=vLLM 0.17.1, Hardware=NVIDIA H2002026.05 | 1.22 | — | |
| ARTarget Model=Qwen3-8B, Temperature=02026.05 | 1 | 1 | |
| ARTarget Model=Qwen3-14B, Temperature=02026.05 | 1 | 1 | |
| ARTarget Model=Qwen3-32B, Temperature=02026.05 | 1 | 1 | |
| ARTarget Model=Q3 8B, Temperature=0, Inference draft length=N/A2026.05 | 1 | 1 | |
| ARTarget Model=Q3 14B, Temperature=0, Inference draft length=N/A2026.05 | 1 | 1 | |
| ARTarget Model=L3.1 8B, Temperature=0, Inference draft length=N/A2026.05 | 1 | 1 | |
| Llama 3.1 8B + EAGLE3Model=Llama 3.1 8B, Decoding Method=SGLang + EAGLE3, Steps=7, Top-k=1, Draft Length=8, Normalization=Pre-norm2026.05 | — | 3.84 | |
| Llama 3.1 8B + EAGLE3Model=Llama 3.1 8B, Decoding Method=SGLang + EAGLE3, Steps=7, Top-k=1, Draft Length=8, Normalization=Post-norm2026.05 | — | 3.9 |