List your company
ResearchFormerly WizWand

Multi-turn Dialogue Generation on MT-bench

5.62Mean Accepted LengthBlockPilot
1.6696642.6952323.72084.746368May 11, 2026May 19, 2026May 27, 2026Jun 5, 2026Jun 13, 2026Jun 21, 2026Jun 30, 2026
Updated 4mo ago

Evaluation results

MethodLinks
BlockPilotModel=Qwen3-4B, Temperature=02026.065.622.85
BlockPilotModel=Qwen3-4B, Temperature=12026.065.593.31
DFlashModel=Qwen3-4B, Temperature=0, Block size (n)=162026.065.292.69
DFlashModel=Qwen3-4B, Temperature=1, Block size (n)=162026.065.233.1
CATS w/ EAGLEDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=tree base, Backbone=Vicuna-7B, Drafting steps=5, top-K=102026.054.52294.52
BlockPilotModel=Qwen3-8B, Temperature=02026.064.372.55
DFlashModel=Qwen3-8B, Temperature=0, Block size (n)=162026.064.252.48
BlockPilotModel=Qwen3-8B, Temperature=12026.064.232.48
DFlashModel=Qwen3-8B, Temperature=0, Block size (n)=322026.064.062.37
DFlashModel=Qwen3-4B, Temperature=0, Block size (n)=82026.063.911.99
DFlashModel=Qwen3-4B, Temperature=1, Block size (n)=82026.063.792.24
DFlashModel=Qwen3-8B, Temperature=1, Block size (n)=322026.063.792.23
CATSDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.053.75973.89
DFlashModel=Qwen3-8B, Temperature=1, Block size (n)=162026.063.752.2
EAGLE-selfDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=tree base, Backbone=Vicuna-7B, Drafting steps=5, top-K=102026.053.72712.82
CATS w/ EAGLEDecoding Strategy=Greedy, Decoding Structure=tree base, Backbone=Vicuna-7B, Drafting steps=5, top-K=102026.053.69383.69
DFlashModel=Qwen3-8B, Temperature=0, Block size (n)=82026.063.452.01
DFlashModel=Qwen3-8B, Temperature=1, Block size (n)=82026.063.281.93
DFlashModel=Qwen3-4B, Temperature=1, Block size (n)=322026.063.221.91
DFlashModel=Qwen3-4B, Temperature=0, Block size (n)=322026.063.191.62
EAGLE-selfDecoding Strategy=Greedy, Decoding Structure=tree base, Backbone=Vicuna-7B, Drafting steps=5, top-K=102026.053.15352.37
EAGLE-selfDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.053.07532.33
KangarooDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.053.07122.35
CATSDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.053.0523.16
EAGLE-3Model=Qwen3-4B, Temperature=02026.063.021.74
EAGLE-3Model=Qwen3-4B, Temperature=12026.062.991.77
DFlashModel=Qwen3-4B, Temperature=0, Block size (n)=42026.062.851.45
EAGLE-3Model=Qwen3-8B, Temperature=02026.062.831.63
DFlashModel=Qwen3-4B, Temperature=1, Block size (n)=42026.062.741.62
EAGLE-3Model=Qwen3-8B, Temperature=12026.062.71.58
MedusaDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base, Backbone=Vicuna-7B, Number of heads=22026.052.68022.69
DFlashModel=Qwen3-8B, Temperature=0, Block size (n)=42026.062.581.51
EAGLE-selfDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.052.51511.9
MedusaDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B, Number of heads=22026.052.51152.52
KangarooDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.052.48631.92
DFlashModel=Qwen3-8B, Temperature=1, Block size (n)=42026.062.451.44
LookaheadDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B2026.052.29282.3
RESTDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B2026.051.82391.86
RESTDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base, Backbone=Vicuna-7B2026.051.82161.86