Code Generation on HumanEval (tau and Speedup)
8.22Speedup FactorDFlash+DDTree
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| DFlash+DDTreeModel=Qwen3-Coder-30B-A3B-Instruct, Temperature=0.0, Node Budget=Best from {16, 32, 64, 128, 256, 512, 1024}2026.04 | 8.22 | 10.72 | — | |
| DFlash+DDTreeModel=Qwen3-Coder-30B-A3B-Instruct, Temperature=1.0, Node Budget=Best from {16, 32, 64, 128, 256, 512, 1024}2026.04 | 7.88 | 10.42 | — | |
| PARD-2Target Model=L3.1 8B, Temperature=0, Inference draft length=162026.05 | 6.94 | 9.19 | — | |
| DFlash+DDTreeModel=Qwen3-8B, Temperature=0.0, Node Budget=Best from {16, 32, 64, 128, 256, 512, 1024}2026.04 | 6.9 | 9.67 | — | |
| DFlash+DDTreeModel=Qwen3-4B, Temperature=0.0, Node Budget=Best from {16, 32, 64, 128, 256, 512, 1024}2026.04 | 6.81 | 9.44 | — | |
| PARD-2Target Model=Q3 8B, Temperature=0, Inference draft length=162026.05 | 6.75 | 8.09 | — | |
| PARD-2Target Model=Q3 14B, Temperature=0, Inference draft length=162026.05 | 6.74 | 7.95 | — | |
| DFlash+DDTreeModel=Qwen3-4B, Temperature=1.0, Node Budget=Best from {16, 32, 64, 128, 256, 512, 1024}2026.04 | 6.43 | 9.18 | — | |
| DFlashModel=Qwen3-Coder-30B-A3B-Instruct, Temperature=0.02026.04 | 6.09 | 8.02 | — | |
| DFlash+DDTreeModel=Qwen3-8B, Temperature=1.0, Node Budget=Best from {16, 32, 64, 128, 256, 512, 1024}2026.04 | 6.09 | 8.59 | — | |
| Draft-OPDModel=Q3-8B, Thinking Mode=Disabled, Temperature=02026.05 | 6.02 | 6.57 | — | |
| DFlashTarget Model=Q3 8B, Temperature=0, Inference draft length=162026.05 | 5.66 | 6.94 | — | |
| Draft-OPDModel=Q3-4B, Thinking Mode=Disabled, Temperature=02026.05 | 5.65 | 6.6 | — | |
| DFlashModel=Qwen3-Coder-30B-A3B-Instruct, Temperature=1.02026.04 | 5.64 | 7.6 | — | |
| DFlashModel=Q3-8B, Thinking Mode=Disabled, Temperature=02026.05 | 5.64 | 6.04 | — | |
| Draft-OPDModel=Q3-4B, Thinking Mode=Disabled, Temperature=0.62026.05 | 5.42 | 6.13 | — | |
| Draft-OPDModel=Q3-8B, Thinking Mode=Disabled, Temperature=0.62026.05 | 5.3 | 6.02 | — | |
| Draft-OPDModel=Q3-8B, Thinking Mode=Enabled, Temperature=02026.05 | 5.19 | 5.73 | — | |
| DFlashModel=Q3-4B, Thinking Mode=Disabled, Temperature=02026.05 | 5.19 | 6.04 | — | |
| Draft-OPDModel=Q3-4B, Thinking Mode=Enabled, Temperature=02026.05 | 5.17 | 5.96 | — | |
| DFlashModel=Q3-4B, Thinking Mode=Disabled, Temperature=0.62026.05 | 5.17 | 5.73 | — | |
| PARDTarget Model=Q3 14B, Temperature=0, Inference draft length=162026.05 | 5.16 | 5.25 | — | |
| PARDTarget Model=L3.1 8B, Temperature=0, Inference draft length=162026.05 | 5.09 | 6.71 | — | |
| PARDTarget Model=Q3 8B, Temperature=0, Inference draft length=162026.05 | 5.02 | 6.33 | — | |
| DFlashModel=Q3-8B, Thinking Mode=Disabled, Temperature=0.62026.05 | 4.97 | 5.62 | — | |
| EAGLE-3Model=Q3-8B, Thinking Mode=Disabled, Temperature=02026.05 | 4.94 | 5.99 | — | |
| PPOWTarget Model=Vicuna-7B2026.05 | 4.87 | 6.75 | — | |
| CATS w/ EAGLEDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=tree base, Backbone=Vicuna-7B, Drafting steps=5, top-K=102026.05 | 4.85 | 4.8408 | — | |
| DFlashModel=Qwen3-8B, Temperature=0.02026.04 | 4.84 | 6.61 | — | |
| DFlashModel=Qwen3-4B, Temperature=0.02026.04 | 4.81 | 6.62 | — | |
| DFlashModel=Q3-4B, Thinking Mode=Enabled, Temperature=02026.05 | 4.77 | 5.51 | — | |
| PPOWTarget Model=LLaMA-2-7B2026.05 | 4.75 | 6.68 | — | |
| DFlashModel=Q3-8B, Thinking Mode=Enabled, Temperature=02026.05 | 4.71 | 5.19 | — | |
| EAGLE-3Model=Q3-4B, Thinking Mode=Disabled, Temperature=02026.05 | 4.53 | 5.84 | — | |
| Draft-OPDModel=Q3-8B, Thinking Mode=Enabled, Temperature=0.62026.05 | 4.47 | 5.05 | — | |
| EAGLE-3Model=Q3-8B, Thinking Mode=Enabled, Temperature=02026.05 | 4.46 | 5.64 | — | |
| Draft-OPDModel=Q3-4B, Thinking Mode=Enabled, Temperature=0.62026.05 | 4.4 | 5.13 | — | |
| EAGLE-3Model=Q3-4B, Thinking Mode=Enabled, Temperature=02026.05 | 4.38 | 5.33 | — | |
| DFlashModel=Qwen3-4B, Temperature=1.02026.04 | 4.34 | 5.97 | — | |
| EAGLE-3Model=Q3-8B, Thinking Mode=Disabled, Temperature=0.62026.05 | 4.29 | 5.69 | — | |
| CATSDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.05 | 4.24 | 4.0802 | — | |
| CATS w/ EAGLEDecoding Strategy=Greedy, Decoding Structure=tree base, Backbone=Vicuna-7B, Drafting steps=5, top-K=102026.05 | 4.21 | 4.2009 | — | |
| DFlashModel=Q3-4B, Thinking Mode=Enabled, Temperature=0.62026.05 | 4.17 | 4.77 | — | |
| PPOWTarget Model=LLaMA-3.1-8B-Instruct2026.05 | 4.14 | 7.23 | — | |
| DFlashModel=Q3-8B, Thinking Mode=Enabled, Temperature=0.62026.05 | 4.12 | 4.66 | — | |
| FastDraftTarget Model=LLaMA-3.1-8B-Instruct2026.05 | 4.05 | 6.99 | — | |
| EAGLE-3Model=Q3-4B, Thinking Mode=Disabled, Temperature=0.62026.05 | 4.02 | 5.66 | — | |
| DFlashModel=Qwen3-8B, Temperature=1.02026.04 | 4 | 5.48 | — | |
| EAGLE-3Model=Q3-8B, Thinking Mode=Enabled, Temperature=0.62026.05 | 3.95 | 5.33 | — | |
| EAGLE-3Model=Q3-4B, Thinking Mode=Enabled, Temperature=0.62026.05 | 3.77 | 5.03 | — | |
| CATSDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.05 | 3.65 | 3.5174 | — | |
| EDA (Ours)Temperature=0, Target Model=Qwen2.5-Coder-7B2026.03 | 3.36 | 5.35 | — | |
| GOOSEModel=Llama-3-8B, GPU=NVIDIA A402026.04 | 3.36 | 4.11 | — | |
| EAGLE-2Model=Vicuna-13B, GPU=NVIDIA A402026.04 | 3.34 | 5.24 | — | |
| EAGLE-3Target Model=L3.1 8B, Temperature=0, Inference draft length=82026.05 | 3.34 | 4.55 | — | |
| EAGLE-selfDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=tree base, Backbone=Vicuna-7B, Drafting steps=5, top-K=102026.05 | 3.14 | 4.1547 | — | |
| Full-FTTemperature=0, Target Model=Qwen2.5-Coder-7B2026.03 | 3.1 | 4.79 | — | |
| EDA (Base)Temperature=0, Target Model=Qwen2.5-Coder-7B2026.03 | 3.08 | 4.75 | — | |
| LoRATemperature=0, Target Model=Qwen2.5-Coder-7B2026.03 | 3.05 | 4.7 | — | |
| EAGLE-2Model=Vicuna-7B, GPU=NVIDIA A402026.04 | 2.99 | 5.02 | — | |
| GOOSEModel=Vicuna-7B, GPU=NVIDIA A402026.04 | 2.97 | 3.63 | — | |
| MedusaDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base, Backbone=Vicuna-7B, Number of heads=22026.05 | 2.89 | 2.876 | — | |
| GOOSEModel=Vicuna-13B, GPU=NVIDIA A402026.04 | 2.84 | 3.37 | — | |
| TRModel=Llama-3-8B, GPU=NVIDIA A402026.04 | 2.79 | 3.4 | — | |
| MedusaDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B, Number of heads=22026.05 | 2.7 | 2.6832 | — | |
| EAGLE-selfDecoding Strategy=Greedy, Decoding Structure=tree base, Backbone=Vicuna-7B, Drafting steps=5, top-K=102026.05 | 2.7 | 3.5949 | — | |
| EAGLE-selfDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.05 | 2.61 | 3.4546 | — | |
| KangarooDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.05 | 2.59 | 3.3703 | — | |
| TRModel=Vicuna-7B, GPU=NVIDIA A402026.04 | 2.53 | 3.06 | — | |
| LookaheadDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B2026.05 | 2.53 | 2.5128 | — | |
| TRModel=Vicuna-13B, GPU=NVIDIA A402026.04 | 2.49 | 2.94 | — | |
| LogitSpecBackbone=Vicuna 13B2025.07 | 2.47 | — | 2.88 | |
| EAGLE-2Model=Llama-3-8B, GPU=NVIDIA A402026.04 | 2.46 | 4.95 | — | |
| GOOSEModel=Vicuna-33B, GPU=NVIDIA A1002026.04 | 2.37 | 2.83 | — | |
| GOOSEModel=Qwen2-8B, GPU=NVIDIA A402026.04 | 2.34 | 2.78 | — | |
| EAGLE-3Target Model=Q3 14B, Temperature=0, Inference draft length=82026.05 | 2.32 | 2.86 | — | |
| LogitSpecModel=Llama-3.1-8B-Instruct2025.07 | 2.31 | — | 2.63 | |
| EDA (Ours)Temperature=1, Target Model=Qwen2.5-Coder-7B2026.03 | 2.28 | 4.38 | — | |
| LogitSpecBackbone=Vicuna 7B2025.07 | 2.24 | — | 2.62 | |
| KangarooDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.05 | 2.24 | 2.8979 | — | |
| EAGLE-selfDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B, Drafting steps=52026.05 | 2.19 | 2.9005 | — | |
| TRModel=Qwen2-8B, GPU=NVIDIA A402026.04 | 2.18 | 2.55 | — | |
| EAGLE-3Target Model=Q3 8B, Temperature=0, Inference draft length=82026.05 | 2.11 | 2.68 | — | |
| LogitSpecModels=Llama 2 13B2025.07 | 2.1 | — | 2.38 | |
| Full-FTTemperature=1, Target Model=Qwen2.5-Coder-7B2026.03 | 2 | 3.72 | — | |
| EDA (Base)Temperature=1, Target Model=Qwen2.5-Coder-7B2026.03 | 1.98 | 3.7 | — | |
| LookaheadModel=Llama-3-8B, GPU=NVIDIA A402026.04 | 1.98 | 2.08 | — | |
| TRModel=Vicuna-33B, GPU=NVIDIA A1002026.04 | 1.97 | 2.37 | — | |
| LoRATemperature=1, Target Model=Qwen2.5-Coder-7B2026.03 | 1.95 | 3.6 | — | |
| SpSBackbone=Vicuna 13B2025.07 | 1.95 | — | 2.66 | |
| LogitSpecBackbone=Vicuna 33B2025.07 | 1.95 | — | 2.41 | |
| LogitSpecModels=Llama 2 70B2025.07 | 1.93 | — | 2.33 | |
| PLDModel=Llama-3-8B, GPU=NVIDIA A402026.04 | 1.92 | 2.01 | — | |
| LogitSpecModel=Qwen-3-8B2025.07 | 1.92 | — | 2.18 | |
| RESTDecoding Strategy=Relaxed (temperature = 0.7), Decoding Structure=chain base, Backbone=Vicuna-7B2026.05 | 1.84 | 1.7984 | — | |
| RESTDecoding Strategy=Greedy, Decoding Structure=chain base, Backbone=Vicuna-7B2026.05 | 1.81 | 1.767 | — | |
| SpSBackbone=Vicuna 7B2025.07 | 1.8 | — | 2.56 | |
| SpSBackbone=Vicuna 33B2025.07 | 1.8 | — | 2.27 | |
| PLDBackbone=Vicuna 13B2025.07 | 1.77 | — | 1.98 | |
| SpSModels=Llama 2 13B2025.07 | 1.76 | — | 2.33 |