Code Generation on MBPP (average acceptance length (τ), Speedup)
8.99Average Acceptance Length (τ)BASTION
Evaluation results
| Method | Links | ||
|---|---|---|---|
| BASTIONModel=Qwen3-4B, Decoding Mode=Greedy Decoding, Temperature=02026.05 | 8.99 | 7.41 | |
| BASTIONModel=Qwen3-8B, Decoding Mode=Stochastic Decoding, Temperature=12026.05 | 8.78 | 6.75 | |
| BASTIONModel=Qwen3-4B, Decoding Mode=Stochastic Decoding, Temperature=12026.05 | 8.77 | 7.07 | |
| BASTIONModel=Qwen3-8B, Decoding Mode=Greedy Decoding, Temperature=02026.05 | 8.65 | 7.1 | |
| JetFlowBudget=256, Temperature=02026.06 | 7.43 | 6.73 | |
| DOMINOTemperature=0, Model=Qwen3-4B, Draft size=162026.05 | 7.11 | 5.59 | |
| DOMINOTemperature=0, Model=Qwen3-8B, Draft size=162026.05 | 7.04 | 5.53 | |
| JetFlowBudget=128, Temperature=02026.06 | 6.95 | 6.31 | |
| JetFlowBudget=256, Temperature=12026.06 | 6.83 | 6.11 | |
| DFlash-TBudget=256, Temperature=02026.06 | 6.7 | 6.09 | |
| BlockPilotModel=Qwen3-4B, Temperature=02026.06 | 6.42 | 4.38 | |
| JetFlowBudget=128, Temperature=12026.06 | 6.39 | 5.78 | |
| DOMINOTemperature=1, Model=Qwen3-4B, Draft size=162026.05 | 6.36 | 4.96 | |
| BlockPilotModel=Qwen3-4B, Temperature=12026.06 | 6.33 | 4.59 | |
| DFlashModel=Qwen3-4B, Decoding Mode=Greedy Decoding, Temperature=02026.05 | 6.31 | 5.09 | |
| DFlash-TBudget=128, Temperature=02026.06 | 6.28 | 5.7 | |
| BlockPilotModel=Qwen3-8B, Temperature=02026.06 | 6.26 | 4.53 | |
| DFlashModel=Qwen3-4B, Temperature=0, Block size (n)=162026.06 | 6.24 | 4.26 | |
| BlockPilotModel=Qwen3-8B, Temperature=12026.06 | 6.17 | 4.54 | |
| FlexDraftModel=Qwen3-8B, Scaled training=true2026.05 | 6.16 | 4.46 | |
| DFlashTemperature=0, Model=Qwen3-4B, Draft size=162026.05 | 6.12 | 4.77 | |
| DFlash-TBudget=256, Temperature=12026.06 | 6.12 | 5.49 | |
| DFlashModel=Qwen3-8B, Decoding Mode=Greedy Decoding, Temperature=02026.05 | 6.04 | 4.91 | |
| DOMINOTemperature=1, Model=Qwen3-8B, Draft size=162026.05 | 6.02 | 4.73 | |
| DFlashModel=Qwen3-8B, Temperature=0, Block size (n)=162026.06 | 5.99 | 4.33 | |
| DFlashTemperature=0, Model=Qwen3-8B, Draft size=162026.05 | 5.98 | 4.71 | |
| dFlashModel=Qwen3-8B, Scaled training=true2026.05 | 5.93 | 4.16 | |
| DFlashModel=Qwen3-4B, Decoding Mode=Stochastic Decoding, Temperature=12026.05 | 5.88 | 4.73 | |
| JetFlowBudget=64, Temperature=02026.06 | 5.88 | 5.34 | |
| DFlash-TBudget=128, Temperature=12026.06 | 5.75 | 5.22 | |
| DFlashTemperature=1, Model=Qwen3-4B, Draft size=162026.05 | 5.67 | 4.44 | |
| DFlashModel=Qwen3-4B, Temperature=1, Block size (n)=162026.06 | 5.54 | 4.02 | |
| JetFlowBudget=64, Temperature=12026.06 | 5.51 | 5 | |
| DFlash-TBudget=64, Temperature=02026.06 | 5.49 | 4.99 | |
| DFlashModel=Qwen3-8B, Decoding Mode=Stochastic Decoding, Temperature=12026.05 | 5.44 | 4.13 | |
| DFlashModel=Qwen3-8B, Temperature=0, Block size (n)=322026.06 | 5.42 | 3.92 | |
| DFlashModel=Qwen3-8B, Temperature=1, Block size (n)=162026.06 | 5.41 | 3.98 | |
| DFlashTemperature=1, Model=Qwen3-8B, Draft size=162026.05 | 5.23 | 4.14 | |
| DFlash-TBudget=64, Temperature=12026.06 | 5.11 | 4.65 | |
| JetFlowBudget=32, Temperature=02026.06 | 5.03 | 4.03 | |
| DFlashModel=Qwen3-8B, Temperature=1, Block size (n)=322026.06 | 5 | 3.68 | |
| DFlashBudget=16, Temperature=02026.06 | 4.98 | 3.96 | |
| JetFlowBudget=16, Temperature=02026.06 | 4.96 | 3.97 | |
| DFlashModel=Qwen3-8B, Temperature=0, Block size (n)=82026.06 | 4.86 | 3.51 | |
| DFlashModel=Qwen3-4B, Temperature=0, Block size (n)=82026.06 | 4.71 | 3.22 | |
| JetFlowBudget=32, Temperature=12026.06 | 4.51 | 3.6 | |
| DFlashBudget=16, Temperature=12026.06 | 4.43 | 3.52 | |
| DFlashModel=Qwen3-4B, Temperature=1, Block size (n)=82026.06 | 4.42 | 3.21 | |
| JetFlowBudget=16, Temperature=12026.06 | 4.41 | 3.52 | |
| DFlashBudget=32, Temperature=02026.06 | 4.39 | 3.51 | |
| DFlashModel=Qwen3-8B, Temperature=1, Block size (n)=82026.06 | 4.36 | 3.2 | |
| HyperDFlashTemperature=0, Number of drafted steps=6, Mode=Non-thinking2026.06 | 4.01 | 3.02 | |
| DFlashBudget=32, Temperature=12026.06 | 3.97 | 3.17 | |
| DFlashModel=Qwen3-4B, Temperature=0, Block size (n)=322026.06 | 3.97 | 2.71 | |
| HyperDFlashTemperature=1, Number of drafted steps=6, Mode=Non-thinking2026.06 | 3.85 | 2.9 | |
| EAGLE-3Budget=64, Temperature=02026.06 | 3.81 | 2.22 | |
| DFlashModel=Qwen3-4B, Temperature=1, Block size (n)=322026.06 | 3.71 | 2.69 | |
| EAGLE-3Budget=64, Temperature=12026.06 | 3.7 | 2.13 | |
| EAGLE-3Budget=32, Temperature=02026.06 | 3.6 | 2.09 | |
| EAGLE-3Budget=32, Temperature=12026.06 | 3.5 | 2.02 | |
| EAGLE-3Temperature=0, Model=Qwen3-4B, Draft size=602026.05 | 3.44 | 2.3 | |
| EAGLE-3Temperature=1, Model=Qwen3-4B, Draft size=602026.05 | 3.38 | 2.19 | |
| EAGLE-3Model=Qwen3-8B, Scaled training=true2026.05 | 3.31 | 2.24 | |
| EAGLE-3Temperature=0, Model=Qwen3-8B, Draft size=602026.05 | 3.31 | 2.22 | |
| EAGLE-3Budget=16, Temperature=02026.06 | 3.31 | 1.95 | |
| EAGLE-3Temperature=1, Model=Qwen3-8B, Draft size=602026.05 | 3.24 | 2.11 | |
| EAGLE-3Budget=16, Temperature=12026.06 | 3.22 | 1.89 | |
| DFlashModel=Qwen3-8B, Temperature=0, Block size (n)=42026.06 | 3.21 | 2.32 | |
| DFlashModel=Qwen3-4B, Temperature=0, Block size (n)=42026.06 | 3.17 | 2.16 | |
| DFlashModel=Qwen3-4B, Temperature=1, Block size (n)=42026.06 | 3.1 | 2.25 | |
| MTPTemperature=0, Number of drafted steps=6, Mode=Non-thinking2026.06 | 3.05 | 1.71 | |
| DFlashModel=Qwen3-8B, Temperature=1, Block size (n)=42026.06 | 3.03 | 2.23 | |
| DARTTemperature=0, Model=Qwen3-4B, Draft size=602026.05 | 3.01 | 2.47 | |
| DARTTemperature=1, Model=Qwen3-4B, Draft size=602026.05 | 3.01 | 2.46 | |
| EAGLE-3Temperature=0, Model=Qwen3-4B, Draft size=162026.05 | 2.99 | 2.02 | |
| DARTModel=Qwen3-8B, Scaled training=true2026.05 | 2.98 | 2.39 | |
| DARTTemperature=0, Model=Qwen3-8B, Draft size=602026.05 | 2.98 | 2.39 | |
| DARTTemperature=1, Model=Qwen3-8B, Draft size=602026.05 | 2.98 | 2.45 | |
| MTPTemperature=1, Number of drafted steps=6, Mode=Non-thinking2026.06 | 2.96 | 1.61 | |
| EAGLE-3Temperature=1, Model=Qwen3-4B, Draft size=162026.05 | 2.95 | 1.97 | |
| EAGLE-3Temperature=1, Model=Qwen3-8B, Draft size=162026.05 | 2.95 | 1.97 | |
| EAGLE-3Model=Qwen3-4B, Temperature=02026.06 | 2.95 | 1.78 | |
| MTPTemperature=0, Number of drafted steps=3, Mode=Non-thinking2026.06 | 2.92 | 2.24 | |
| EAGLE-3Model=Qwen3-4B, Temperature=12026.06 | 2.89 | 1.69 | |
| EAGLE-3Temperature=0, Model=Qwen3-8B, Draft size=162026.05 | 2.86 | 1.93 | |
| MTPTemperature=1, Number of drafted steps=3, Mode=Non-thinking2026.06 | 2.84 | 2.18 | |
| EAGLE-3Model=Qwen3-8B, Temperature=02026.06 | 2.82 | 1.69 | |
| EAGLE-3Model=Qwen3-8B, Temperature=12026.06 | 2.74 | 1.64 | |
| EAGLE-3Model=Qwen3-4B, Decoding Mode=Greedy Decoding, Temperature=02026.05 | 2.48 | 2.77 | |
| Apple MTPModel=Qwen3-8B, Scaled training=true2026.05 | 2.44 | 2.04 | |
| EAGLE-3Model=Qwen3-4B, Decoding Mode=Stochastic Decoding, Temperature=12026.05 | 2.41 | 2.7 | |
| EAGLE-3Model=Qwen3-8B, Decoding Mode=Greedy Decoding, Temperature=02026.05 | 2.24 | 2.73 | |
| EAGLE-3Model=Qwen3-8B, Decoding Mode=Stochastic Decoding, Temperature=12026.05 | 2.23 | 2.52 | |
| Vanilla DFlashTemperature=0, Number of drafted steps=6, Mode=Non-thinking2026.06 | 2.16 | 1.7 | |
| Vanilla DFlashTemperature=1, Number of drafted steps=6, Mode=Non-thinking2026.06 | 2.12 | 1.66 |