List your company
ResearchFormerly WizWand

Code Generation on MBPP (tau, Speedup)

7.68SpeedupDFlash+DDTree
0.51442.37474.2356.0953Mar 10, 2026Mar 27, 2026Apr 14, 2026May 2, 2026May 20, 2026Jun 7, 2026Jun 25, 2026
Updated 4mo ago

Evaluation results

MethodLinks
DFlash+DDTreeModel=Qwen3-Coder-30B-A3B-Instruct, Temperature=0.0, Node Budget=Best from {16, 32, 64, 128, 256, 512, 1024}2026.047.689.94——
DFlash+DDTreeModel=Qwen3-Coder-30B-A3B-Instruct, Temperature=1.0, Node Budget=Best from {16, 32, 64, 128, 256, 512, 1024}2026.047.559.76——
DFlash+DDTreeModel=Qwen3-4B, Temperature=0.0, Node Budget=Best from {16, 32, 64, 128, 256, 512, 1024}2026.046.429.16——
DFlash+DDTreeModel=Qwen3-8B, Temperature=0.0, Node Budget=Best from {16, 32, 64, 128, 256, 512, 1024}2026.046.399.07——
DFlash+DDTreeModel=Qwen3-4B, Temperature=1.0, Node Budget=Best from {16, 32, 64, 128, 256, 512, 1024}2026.046.098.72——
DFlash+DDTreeModel=Qwen3-8B, Temperature=1.0, Node Budget=Best from {16, 32, 64, 128, 256, 512, 1024}2026.045.878.34——
Draft-OPDModel=Q3-8B, Thinking Mode=Disabled, Temperature=02026.055.64——6.57
DFlashModel=Qwen3-Coder-30B-A3B-Instruct, Temperature=0.02026.045.617.19——
DFlashModel=Qwen3-Coder-30B-A3B-Instruct, Temperature=1.02026.045.477.03——
Draft-OPDModel=Q3-4B, Thinking Mode=Disabled, Temperature=02026.055.4——6.6
DFlashModel=Q3-8B, Thinking Mode=Disabled, Temperature=02026.055.17——6.04
Draft-OPDModel=Q3-8B, Thinking Mode=Disabled, Temperature=0.62026.055.17——6.02
DFlashModel=Q3-4B, Thinking Mode=Disabled, Temperature=02026.055——6.04
Draft-OPDModel=Q3-4B, Thinking Mode=Disabled, Temperature=0.62026.054.98——6.13
Draft-OPDModel=Q3-8B, Thinking Mode=Enabled, Temperature=02026.054.86——5.73
Draft-OPDModel=Q3-4B, Thinking Mode=Enabled, Temperature=02026.054.85——5.96
EAGLE-3Model=Q3-8B, Thinking Mode=Disabled, Temperature=02026.054.83——5.99
DFlashModel=Q3-8B, Thinking Mode=Disabled, Temperature=0.62026.054.74——5.62
DFlashModel=Q3-4B, Thinking Mode=Disabled, Temperature=0.62026.054.72——5.73
EAGLE-3Model=Q3-8B, Thinking Mode=Enabled, Temperature=02026.054.55——5.64
EAGLE-3Model=Q3-4B, Thinking Mode=Disabled, Temperature=02026.054.55——5.84
DFlashModel=Q3-8B, Thinking Mode=Enabled, Temperature=02026.054.5——5.19
EAGLE-3Model=Q3-8B, Thinking Mode=Disabled, Temperature=0.62026.054.4——5.69
DFlashModel=Q3-4B, Thinking Mode=Enabled, Temperature=02026.054.39——5.51
DFlashModel=Qwen3-4B, Temperature=0.02026.044.386.1——
DFlashModel=Qwen3-8B, Temperature=0.02026.044.335.99——
EAGLE-3Model=Q3-4B, Thinking Mode=Enabled, Temperature=02026.054.29——5.33
Draft-OPDModel=Q3-4B, Thinking Mode=Enabled, Temperature=0.62026.054.21——5.13
Draft-OPDModel=Q3-8B, Thinking Mode=Enabled, Temperature=0.62026.054.17——5.05
DFlashModel=Qwen3-4B, Temperature=1.02026.044.035.56——
EAGLE-3Model=Q3-4B, Thinking Mode=Disabled, Temperature=0.62026.053.98——5.66
EAGLE-3Model=Q3-8B, Thinking Mode=Enabled, Temperature=0.62026.053.93——5.33
DFlashModel=Q3-4B, Thinking Mode=Enabled, Temperature=0.62026.053.92——4.77
DFlashModel=Q3-8B, Thinking Mode=Enabled, Temperature=0.62026.053.85——4.66
DFlashModel=Qwen3-8B, Temperature=1.02026.043.835.3——
EAGLE-3Model=Q3-4B, Thinking Mode=Enabled, Temperature=0.62026.053.75——5.03
EAGLE-2Model=Vicuna-13B, GPU=NVIDIA A402026.043.415.3——
EDA (Ours)Temperature=0, Target Model=Qwen2.5-Coder-7B2026.033.185.43——
GOOSEModel=Llama-3-8B, GPU=NVIDIA A402026.0433.61——
EAGLE-2Model=Vicuna-7B, GPU=NVIDIA A402026.042.924.83——
GOOSEModel=Vicuna-7B, GPU=NVIDIA A402026.042.843.4——
Full-FTTemperature=0, Target Model=Qwen2.5-Coder-7B2026.032.824.93——
EDA (Base)Temperature=0, Target Model=Qwen2.5-Coder-7B2026.032.84.9——
LoRATemperature=0, Target Model=Qwen2.5-Coder-7B2026.032.764.78——
GOOSEModel=Qwen2-8B, GPU=NVIDIA A402026.042.73.2——
GOOSEModel=Vicuna-13B, GPU=NVIDIA A402026.042.663.11——
TRModel=Llama-3-8B, GPU=NVIDIA A402026.042.63.11——
HyperDFlashTemperature=0, Drafted steps per verification round=6, Decoding mode=Think-high mode2026.062.563.35——
EDA (Ours)Temperature=1, Target Model=Qwen2.5-Coder-7B2026.032.484.92——
TRModel=Qwen2-8B, GPU=NVIDIA A402026.042.432.81——
TRModel=Vicuna-7B, GPU=NVIDIA A402026.042.412.86——
EAGLE-2Model=Llama-3-8B, GPU=NVIDIA A402026.042.414.83——
TRModel=Vicuna-13B, GPU=NVIDIA A402026.042.42.78——
GOOSEModel=Vicuna-33B, GPU=NVIDIA A1002026.042.352.74——
Full-FTTemperature=1, Target Model=Qwen2.5-Coder-7B2026.032.234.32——
EDA (Base)Temperature=1, Target Model=Qwen2.5-Coder-7B2026.032.224.3——
HyperDFlashTemperature=1, Drafted steps per verification round=6, Decoding mode=Think-high mode2026.062.192.87——
LoRATemperature=1, Target Model=Qwen2.5-Coder-7B2026.032.184.25——
MTPTemperature=0, Drafted steps per verification round=3, Decoding mode=Think-high mode2026.062.092.71——
TRModel=Vicuna-33B, GPU=NVIDIA A1002026.041.952.29——
MTPTemperature=1, Drafted steps per verification round=3, Decoding mode=Think-high mode2026.061.912.48——
LookaheadModel=Llama-3-8B, GPU=NVIDIA A402026.041.791.86——
LookaheadModel=Vicuna-7B, GPU=NVIDIA A402026.041.751.78——
LookaheadModel=Qwen2-8B, GPU=NVIDIA A402026.041.751.68——
PLDModel=Llama-3-8B, GPU=NVIDIA A402026.041.671.72——
LookaheadModel=Vicuna-13B, GPU=NVIDIA A402026.041.641.67——
PLDModel=Vicuna-7B, GPU=NVIDIA A402026.041.591.62——
Vanilla DFlashTemperature=0, Drafted steps per verification round=6, Decoding mode=Think-high mode2026.061.561.98——
MTPTemperature=0, Drafted steps per verification round=6, Decoding mode=Think-high mode2026.061.532.82——
LookaheadModel=Vicuna-33B, GPU=NVIDIA A1002026.041.511.49——
PLDModel=Vicuna-13B, GPU=NVIDIA A402026.041.491.51——
Vanilla DFlashTemperature=1, Drafted steps per verification round=6, Decoding mode=Think-high mode2026.061.461.87——
PLDModel=Qwen2-8B, GPU=NVIDIA A402026.041.451.5——
MTPTemperature=1, Drafted steps per verification round=6, Decoding mode=Think-high mode2026.061.422.54——
PLDModel=Vicuna-33B, GPU=NVIDIA A1002026.041.271.3——
Training-FreeTemperature=0, Target Model=Qwen2.5-Coder-7B2026.031.211.85——
RESTModel=Qwen2-8B, GPU=NVIDIA A402026.041.111.04——
RESTModel=Vicuna-33B, GPU=NVIDIA A1002026.041.081.05——
RESTModel=Vicuna-7B, GPU=NVIDIA A402026.041.051.05——
RESTModel=Vicuna-13B, GPU=NVIDIA A402026.041.041.04——
RESTModel=Llama-3-8B, GPU=NVIDIA A402026.041.011.01——
ARModel=Vicuna-7B, GPU=NVIDIA A402026.0411——
ARModel=Llama-3-8B, GPU=NVIDIA A402026.0411——
ARModel=Qwen2-8B, GPU=NVIDIA A402026.0411——
ARModel=Vicuna-13B, GPU=NVIDIA A402026.0411——
ARModel=Vicuna-33B, GPU=NVIDIA A1002026.0411——
Training-FreeTemperature=1, Target Model=Qwen2.5-Coder-7B2026.030.791.3——
D-PACETarget=Llama-3.1-8B-Instruct, Temperature (T)=0, Drafter=3L, Epoch=62026.05—3.682.84—
D-PACETarget=Qwen3-8B, Temperature (T)=0, Drafter=3L, Epoch=62026.05—3.562.65—
DFlashTarget=Llama-3.1-8B-Instruct, Temperature (T)=0, Drafter=3L, Epoch=62026.05—3.282.53—
DFlashTarget=Qwen3-8B, Temperature (T)=0, Drafter=3L, Epoch=62026.05—3.222.4—
FP16Model=Llama-3.2-1B-Instruct2025.11——33—
IntAttentionModel=Llama-3.2-1B-Instruct2025.11——34.2—
MDMSampling steps=256, Zero-shot=true, Backbone=LLaDA-8B-Instruct2025.10——0.182—
MDMSampling steps=512, Zero-shot=true, Backbone=LLaDA-8B-Instruct2025.10——0.278—
MDMSampling steps=1024, Zero-shot=true, Backbone=LLaDA-8B-Instruct2025.10——0.319—
PRISMSampling steps=256, Zero-shot=true, Backbone=LLaDA-8B-Instruct, Trainable parameters=≈ 250M2025.10——0.218—
PRISMSampling steps=512, Zero-shot=true, Backbone=LLaDA-8B-Instruct, Trainable parameters=≈ 250M2025.10——0.291—
PRISMSampling steps=1024, Zero-shot=true, Backbone=LLaDA-8B-Instruct, Trainable parameters=≈ 250M2025.10——0.323—
Quant-OnlyModel=Llama-3.2-1B-Instruct2025.11——31.4—