Multi-turn Dialogue on MT-Bench (Quality and Speed)
4.1SpeedupDOUBLE
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| DOUBLEBase Model=Deepseek-33B2026.01 | 4.1 | — | 9.85 | |
| EAGLE3Base Model=LLaMA3.3-70B2026.01 | 4.02 | — | 5.23 | |
| DOUBLEBase Model=LLaMA3.3-70B2026.01 | 3.88 | — | 9.35 | |
| EAGLE3Base Model=LLaMA3.3-70B, Temperature=1, Decoding Setting=non-greedy2026.01 | 3.74 | 4.86 | — | |
| DOUBLEBase Model=LLaMA3.3-70B, Temperature=1, Decoding Setting=non-greedy2026.01 | 3.42 | 8.25 | — | |
| PEARLBase Model=LLaMA3.3-70B2026.01 | 3.26 | — | 8.38 | |
| PEARLBase Model=Deepseek-33B2026.01 | 3.23 | — | 7.92 | |
| DOUBLEBase Model=LLaMA2-70B2026.01 | 3.22 | — | 8.35 | |
| PEARLBase Model=LLaMA3.3-70B, Temperature=1, Decoding Setting=non-greedy2026.01 | 3.07 | 7.89 | — | |
| GOOSEModel=Qwen2-8B, GPU=NVIDIA A402026.04 | 2.92 | 3.51 | — | |
| PEARLBase Model=LLaMA2-70B2026.01 | 2.87 | — | 6.98 | |
| DFlashModel=Q3-4B, Temperature=0, Diffusion block size=162026.02 | 2.85 | 4.35 | — | |
| GOOSEModel=Llama-3-8B, GPU=NVIDIA A402026.04 | 2.84 | 3.5 | — | |
| EAGLE-2Model=Vicuna-13B, GPU=NVIDIA A402026.04 | 2.78 | 4.33 | — | |
| DFlashModel=Q3-8B, Temperature=0, Diffusion block size=162026.02 | 2.75 | 4.24 | — | |
| DFlashModel=Q3-4B, Temperature=1, Diffusion block size=162026.02 | 2.67 | 4.07 | — | |
| EAGLE-2Model=Vicuna-7B, GPU=NVIDIA A402026.04 | 2.53 | 4.21 | — | |
| DFlashModel=Q3-8B, Temperature=1, Diffusion block size=162026.02 | 2.47 | 3.8 | — | |
| TRModel=Qwen2-8B, GPU=NVIDIA A402026.04 | 2.44 | 2.86 | — | |
| SpsBase Model=LLaMA2-70B2026.01 | 2.43 | — | 4.32 | |
| GOOSEModel=Vicuna-7B, GPU=NVIDIA A402026.04 | 2.42 | 2.88 | — | |
| OuroborosBase Model=Deepseek-33B2026.01 | 2.36 | — | 5.12 | |
| GOOSEModel=Vicuna-13B, GPU=NVIDIA A402026.04 | 2.3 | 2.67 | — | |
| TRModel=Llama-3-8B, GPU=NVIDIA A402026.04 | 2.29 | 2.82 | — | |
| EAGLE3Base Model=Qwen3-32B2026.01 | 2.23 | — | 3.03 | |
| TRModel=Vicuna-7B, GPU=NVIDIA A402026.04 | 2.19 | 2.6 | — | |
| TRModel=Vicuna-13B, GPU=NVIDIA A402026.04 | 2.12 | 2.46 | — | |
| EAGLE3Base Model=Qwen3-32B, Temperature=1, Decoding Setting=non-greedy2026.01 | 2.07 | 2.81 | — | |
| EAGLE-3Model=Q3-4B, Temperature=0, Draft tree size=602026.02 | 2.04 | 3.49 | — | |
| SpsBase Model=Deepseek-33B2026.01 | 2.01 | — | 3.92 | |
| SpsBase Model=LLaMA3.3-70B2026.01 | 2.01 | — | 4.3 | |
| DOUBLEBase Model=Qwen3-32B2026.01 | 1.94 | — | 5.83 | |
| EAGLE-2Model=Llama-3-8B, GPU=NVIDIA A402026.04 | 1.92 | 3.85 | — | |
| EAGLE-3Model=Q3-4B, Temperature=1, Draft tree size=602026.02 | 1.91 | 3.36 | — | |
| EAGLE-3Model=Q3-8B, Temperature=0, Draft tree size=602026.02 | 1.9 | 3.26 | — | |
| EAGLE3Base Model=Qwen3-14B2026.01 | 1.89 | — | 2.83 | |
| GOOSEModel=Vicuna-33B, GPU=NVIDIA A1002026.04 | 1.87 | 2.15 | — | |
| PEARLBase Model=Qwen3-32B2026.01 | 1.78 | — | 3.68 | |
| LookaheadModel=Qwen2-8B, GPU=NVIDIA A402026.04 | 1.78 | 1.74 | — | |
| OuroborosBase Model=LLaMA2-70B2026.01 | 1.76 | — | 4.82 | |
| EAGLE-3Model=Q3-4B, Temperature=0, Draft tree size=162026.02 | 1.74 | 3.02 | — | |
| LookaheadModel=Llama-3-8B, GPU=NVIDIA A402026.04 | 1.73 | 1.8 | — | |
| DOUBLEBase Model=Qwen3-32B, Temperature=1, Decoding Setting=non-greedy2026.01 | 1.72 | 5.15 | — | |
| TRBase Model=Deepseek-33B2026.01 | 1.72 | — | 2.36 | |
| TRModel=Vicuna-33B, GPU=NVIDIA A1002026.04 | 1.72 | 2.01 | — | |
| EAGLE-3Model=Q3-4B, Temperature=1, Draft tree size=162026.02 | 1.7 | 2.95 | — | |
| EAGLE-3Model=Q3-8B, Temperature=1, Draft tree size=602026.02 | 1.7 | 3.05 | — | |
| PEARLBase Model=Qwen3-32B, Temperature=1, Decoding Setting=non-greedy2026.01 | 1.67 | 3.45 | — | |
| TRBase Model=LLaMA2-70B2026.01 | 1.67 | — | 1.75 | |
| PLDModel=Llama-3-8B, GPU=NVIDIA A402026.04 | 1.67 | 1.72 | — | |
| PLDModel=Qwen2-8B, GPU=NVIDIA A402026.04 | 1.67 | 1.73 | — | |
| EAGLE-3Model=Q3-8B, Temperature=0, Draft tree size=162026.02 | 1.63 | 2.83 | — | |
| EAGLE-3Model=Q3-8B, Temperature=1, Draft tree size=162026.02 | 1.58 | 2.7 | — | |
| LookaheadModel=Vicuna-7B, GPU=NVIDIA A402026.04 | 1.58 | 1.6 | — | |
| PLDBase Model=Deepseek-33B2026.01 | 1.47 | — | 1.69 | |
| PLDModel=Vicuna-7B, GPU=NVIDIA A402026.04 | 1.47 | 1.48 | — | |
| LookaheadModel=Vicuna-13B, GPU=NVIDIA A402026.04 | 1.47 | 1.49 | — | |
| PLDBase Model=LLaMA3.3-70B2026.01 | 1.46 | — | 1.59 | |
| LookaheadBase Model=LLaMA2-70B2026.01 | 1.45 | — | 1.72 | |
| PLDBase Model=LLaMA2-70B2026.01 | 1.45 | — | 1.49 | |
| LookaheadBase Model=Deepseek-33B2026.01 | 1.36 | — | 1.65 | |
| PLDBase Model=Qwen3-14B2026.01 | 1.35 | — | 1.47 | |
| PLDModel=Vicuna-13B, GPU=NVIDIA A402026.04 | 1.35 | 1.36 | — | |
| SpsBase Model=Qwen3-32B2026.01 | 1.34 | — | 3.47 | |
| DOUBLEBase Model=Qwen3-14B2026.01 | 1.32 | — | 5.13 | |
| PLDBase Model=Qwen3-32B2026.01 | 1.29 | — | 1.43 | |
| LookaheadModel=Vicuna-33B, GPU=NVIDIA A1002026.04 | 1.26 | 1.23 | — | |
| SpsBase Model=Qwen3-14B2026.01 | 1.21 | — | 3.17 | |
| PEARLBase Model=Qwen3-14B2026.01 | 1.21 | — | 3.77 | |
| PLDModel=Vicuna-33B, GPU=NVIDIA A1002026.04 | 1.14 | 1.15 | — | |
| RESTModel=Qwen2-8B, GPU=NVIDIA A402026.04 | 1.11 | 1.03 | — | |
| RESTModel=Vicuna-13B, GPU=NVIDIA A402026.04 | 1.11 | 1.11 | — | |
| RESTModel=Vicuna-33B, GPU=NVIDIA A1002026.04 | 1.09 | 1.05 | — | |
| RESTModel=Vicuna-7B, GPU=NVIDIA A402026.04 | 1.08 | 1.08 | — | |
| RESTModel=Llama-3-8B, GPU=NVIDIA A402026.04 | 1.05 | 1.05 | — | |
| ARModel=Vicuna-7B, GPU=NVIDIA A402026.04 | 1 | 1 | — | |
| ARModel=Llama-3-8B, GPU=NVIDIA A402026.04 | 1 | 1 | — | |
| ARModel=Qwen2-8B, GPU=NVIDIA A402026.04 | 1 | 1 | — | |
| ARModel=Vicuna-13B, GPU=NVIDIA A402026.04 | 1 | 1 | — | |
| ARModel=Vicuna-33B, GPU=NVIDIA A1002026.04 | 1 | 1 | — |