Multi-turn Dialogue on MT-Bench (Speedup, Acceptance Length, Drafting Cost Share)
3.22SpeedupEAGLE-3
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| EAGLE-3Model=Llama-3.1-8B, Temperature=02026.05 | 3.22 | 6.16 | 31 | |
| Draft-OPDModel=Q3-4B, Thinking Mode=Enabled, Temperature=02026.05 | 3.18 | 5.96 | — | |
| Draft-OPDModel=Q3-8B, Thinking Mode=Enabled, Temperature=02026.05 | 3.18 | 5.73 | — | |
| Draft-OPDModel=Q3-8B, Thinking Mode=Disabled, Temperature=02026.05 | 3.12 | 6.57 | — | |
| SpecBlockModel=Llama-3.1-8B, Temperature=02026.05 | 3.1 | 4.46 | 16 | |
| Draft-OPDModel=Q3-4B, Thinking Mode=Disabled, Temperature=02026.05 | 3.09 | 6.6 | — | |
| EAGLE-3Model=Q3-4B, Thinking Mode=Enabled, Temperature=02026.05 | 3.07 | 5.33 | — | |
| EAGLE-3Model=Q3-8B, Thinking Mode=Disabled, Temperature=02026.05 | 3.06 | 5.99 | — | |
| EAGLE-3Model=Q3-8B, Thinking Mode=Enabled, Temperature=02026.05 | 3.02 | 5.64 | — | |
| DFlashModel=Q3-8B, Thinking Mode=Enabled, Temperature=02026.05 | 3.01 | 5.19 | — | |
| DFlashModel=Q3-4B, Thinking Mode=Disabled, Temperature=02026.05 | 3.01 | 6.04 | — | |
| Draft-OPDModel=Q3-8B, Thinking Mode=Disabled, Temperature=0.62026.05 | 2.97 | 6.02 | — | |
| DFlashModel=Q3-4B, Thinking Mode=Enabled, Temperature=02026.05 | 2.96 | 5.51 | — | |
| DFlashModel=Q3-8B, Thinking Mode=Disabled, Temperature=02026.05 | 2.92 | 6.04 | — | |
| Draft-OPDModel=Q3-8B, Thinking Mode=Enabled, Temperature=0.62026.05 | 2.91 | 5.05 | — | |
| Draft-OPDModel=Q3-4B, Thinking Mode=Enabled, Temperature=0.62026.05 | 2.87 | 5.13 | — | |
| Draft-OPDModel=Q3-4B, Thinking Mode=Disabled, Temperature=0.62026.05 | 2.86 | 6.13 | — | |
| EAGLE-3Model=Q3-8B, Thinking Mode=Enabled, Temperature=0.62026.05 | 2.83 | 5.33 | — | |
| DFlashModel=Q3-8B, Thinking Mode=Disabled, Temperature=0.62026.05 | 2.81 | 5.62 | — | |
| EAGLE-3Model=Q3-4B, Thinking Mode=Disabled, Temperature=02026.05 | 2.8 | 5.84 | — | |
| DFlashModel=Q3-4B, Thinking Mode=Disabled, Temperature=0.62026.05 | 2.8 | 5.73 | — | |
| EAGLE-3Model=Q3-4B, Thinking Mode=Enabled, Temperature=0.62026.05 | 2.77 | 5.03 | — | |
| DFlashModel=Q3-8B, Thinking Mode=Enabled, Temperature=0.62026.05 | 2.76 | 4.66 | — | |
| DFlashModel=Q3-4B, Thinking Mode=Enabled, Temperature=0.62026.05 | 2.73 | 4.77 | — | |
| EAGLE-3Model=Q3-8B, Thinking Mode=Disabled, Temperature=0.62026.05 | 2.7 | 5.69 | — | |
| FalconModel=Llama-3.1-8B, Temperature=02026.05 | 2.66 | 4.31 | 26 | |
| EAGLE-3Model=Q3-4B, Thinking Mode=Disabled, Temperature=0.62026.05 | 2.5 | 5.66 | — | |
| EAGLE-3Model=Qwen3-8B, Temperature=02026.05 | 2.4 | 4.23 | 29 | |
| ParallelSpecModel=Llama-3.1-8B, Temperature=02026.05 | 2.36 | 3.21 | 8 | |
| SpecBlockModel=Qwen3-8B, Temperature=02026.05 | 2.33 | 3.32 | 14 | |
| SpecBlockModel=Qwen3-32B, Temperature=02026.05 | 2.21 | 3.21 | 11 | |
| EAGLE-3Model=Qwen3-32B, Temperature=02026.05 | 2.18 | 4.09 | 24 | |
| SpecBlockModel=Qwen3-8B, Temperature=1.02026.05 | 2.06 | 3.19 | 14 | |
| SpecBlockModel=Qwen3-32B, Temperature=1.02026.05 | 1.99 | 3.09 | 11 | |
| EAGLE-3Model=Qwen3-8B, Temperature=1.02026.05 | 1.93 | 3.93 | 29 | |
| MedusaModel=Llama-3.1-8B, Temperature=02026.05 | 1.88 | 2.59 | 6 | |
| EAGLE-3Model=Qwen3-32B, Temperature=1.02026.05 | 1.85 | 3.82 | 25 | |
| SpecBlockModel=Llama-3.1-8B, Temperature=1.02026.05 | 1.5 | 3.46 | 14 | |
| EAGLE-3Model=Llama-3.1-8B, Temperature=1.02026.05 | 1.48 | 4.25 | 30 | |
| SpSModel=Llama-3.1-8B, Temperature=02026.05 | 1.38 | 2.45 | 38 | |
| FalconModel=Llama-3.1-8B, Temperature=1.02026.05 | 1.27 | 3.55 | 27 | |
| ParallelSpecModel=Llama-3.1-8B, Temperature=1.02026.05 | 1.06 | 2.67 | 9 | |
| MedusaModel=Llama-3.1-8B, Temperature=1.02026.05 | 0.9 | 2.13 | 7 | |
| SpSModel=Llama-3.1-8B, Temperature=1.02026.05 | 0.63 | 2.01 | 40 |