Multi-turn Dialogue Generation on MT-bench
5.62Mean Accepted LengthBlockPilot
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| BlockPilotModel=Qwen3-4B, Temperature=02026.06 | 5.62 | 2.85 | |
| BlockPilotModel=Qwen3-4B, Temperature=12026.06 | 5.59 | 3.31 | |
| DFlashModel=Qwen3-4B, Temperature=0, Block size (n)=162026.06 | 5.29 | 2.69 | |
| DFlashModel=Qwen3-4B, Temperature=1, Block size (n)=162026.06 | 5.23 | 3.1 | |
| CATS w/ EAGLEDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=tree base, Backbone=Vicuna-7B, Drafting steps=5, top-K=102026.05 | 4.5229 | 4.52 | |
| BlockPilotModel=Qwen3-8B, Temperature=02026.06 | 4.37 | 2.55 | |
| DFlashModel=Qwen3-8B, Temperature=0, Block size (n)=162026.06 | 4.25 | 2.48 | |
| BlockPilotModel=Qwen3-8B, Temperature=12026.06 | 4.23 | 2.48 | |
| DFlashModel=Qwen3-8B, Temperature=0, Block size (n)=322026.06 | 4.06 | 2.37 | |
| DFlashModel=Qwen3-4B, Temperature=0, Block size (n)=82026.06 | 3.91 | 1.99 | |
| DFlashModel=Qwen3-4B, Temperature=1, Block size (n)=82026.06 | 3.79 | 2.24 | |
| DFlashModel=Qwen3-8B, Temperature=1, Block size (n)=322026.06 | 3.79 | 2.23 | |
| CATSDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.05 | 3.7597 | 3.89 | |
| DFlashModel=Qwen3-8B, Temperature=1, Block size (n)=162026.06 | 3.75 | 2.2 | |
| EAGLE-selfDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=tree base, Backbone=Vicuna-7B, Drafting steps=5, top-K=102026.05 | 3.7271 | 2.82 | |
| CATS w/ EAGLEDecoding Strategy=Greedy, Decoding Structure=tree base, Backbone=Vicuna-7B, Drafting steps=5, top-K=102026.05 | 3.6938 | 3.69 | |
| DFlashModel=Qwen3-8B, Temperature=0, Block size (n)=82026.06 | 3.45 | 2.01 | |
| DFlashModel=Qwen3-8B, Temperature=1, Block size (n)=82026.06 | 3.28 | 1.93 | |
| DFlashModel=Qwen3-4B, Temperature=1, Block size (n)=322026.06 | 3.22 | 1.91 | |
| DFlashModel=Qwen3-4B, Temperature=0, Block size (n)=322026.06 | 3.19 | 1.62 | |
| EAGLE-selfDecoding Strategy=Greedy, Decoding Structure=tree base, Backbone=Vicuna-7B, Drafting steps=5, top-K=102026.05 | 3.1535 | 2.37 | |
| EAGLE-selfDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.05 | 3.0753 | 2.33 | |
| KangarooDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.05 | 3.0712 | 2.35 | |
| CATSDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.05 | 3.052 | 3.16 | |
| EAGLE-3Model=Qwen3-4B, Temperature=02026.06 | 3.02 | 1.74 | |
| EAGLE-3Model=Qwen3-4B, Temperature=12026.06 | 2.99 | 1.77 | |
| DFlashModel=Qwen3-4B, Temperature=0, Block size (n)=42026.06 | 2.85 | 1.45 | |
| EAGLE-3Model=Qwen3-8B, Temperature=02026.06 | 2.83 | 1.63 | |
| DFlashModel=Qwen3-4B, Temperature=1, Block size (n)=42026.06 | 2.74 | 1.62 | |
| EAGLE-3Model=Qwen3-8B, Temperature=12026.06 | 2.7 | 1.58 | |
| MedusaDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base, Backbone=Vicuna-7B, Number of heads=22026.05 | 2.6802 | 2.69 | |
| DFlashModel=Qwen3-8B, Temperature=0, Block size (n)=42026.06 | 2.58 | 1.51 | |
| EAGLE-selfDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.05 | 2.5151 | 1.9 | |
| MedusaDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B, Number of heads=22026.05 | 2.5115 | 2.52 | |
| KangarooDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.05 | 2.4863 | 1.92 | |
| DFlashModel=Qwen3-8B, Temperature=1, Block size (n)=42026.06 | 2.45 | 1.44 | |
| LookaheadDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B2026.05 | 2.2928 | 2.3 | |
| RESTDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B2026.05 | 1.8239 | 1.86 | |
| RESTDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base, Backbone=Vicuna-7B2026.05 | 1.8216 | 1.86 |