List your company
ResearchFormerly WizWand

Code Generation on LCB

7.81SpeedupBASTION
1.08122.82814.5756.3219Feb 5, 2026Feb 28, 2026Mar 23, 2026Apr 16, 2026May 9, 2026Jun 1, 2026Jun 25, 2026
Updated 3mo ago

Evaluation results

MethodLinks
BASTIONModel=Qwen3-4B, Decoding Mode=Greedy Decoding, Temperature=02026.057.81—9.48—
BASTIONModel=Qwen3-8B, Decoding Mode=Stochastic Decoding, Temperature=12026.057.81—9.82—
BASTIONModel=Qwen3-4B, Decoding Mode=Stochastic Decoding, Temperature=12026.057.69—9.83—
JetFlowBudget=256, Temperature=02026.067.67—8.79—
BASTIONModel=Qwen3-8B, Decoding Mode=Greedy Decoding, Temperature=02026.057.64—9.91—
JetFlowBudget=128, Temperature=02026.067.29—8.21—
JetFlowBudget=256, Temperature=12026.067.25—8.29—
JetFlowBudget=128, Temperature=12026.066.84—7.74—
DFlash-TBudget=256, Temperature=02026.066.75—7.72—
DFlash-TBudget=128, Temperature=02026.066.42—7.25—
DFlash-TBudget=256, Temperature=12026.066.26—7.17—
DFlash-TBudget=128, Temperature=12026.065.99—6.71—
JetFlowBudget=64, Temperature=02026.065.95—6.59—
DFlashModel=Qwen3-8B, Decoding Mode=Greedy Decoding, Temperature=02026.055.9—7.64—
JetFlowBudget=64, Temperature=12026.065.76—6.38—
DFlashModel=Qwen3-4B, Decoding Mode=Greedy Decoding, Temperature=02026.055.71—7.23—
DFlashTemperature=0, Model=Qwen3-4B, Draft size=162026.055.48—7.15—
DFlash-TBudget=64, Temperature=02026.065.47—6.06—
DFlashTemperature=0, Model=Qwen3-8B, Draft size=162026.055.37—7.12—
DOMINOTemperature=0, Model=Qwen3-4B, Draft size=162026.055.33—6.86—
DFlash-TBudget=64, Temperature=12026.065.28—5.82—
DOMINOTemperature=0, Model=Qwen3-8B, Draft size=162026.055.27—7.04—
DFlashModel=Qwen3-8B, Decoding Mode=Stochastic Decoding, Temperature=12026.055.21—6.91—
DFlashModel=Qwen3-4B, Decoding Mode=Stochastic Decoding, Temperature=12026.055.1—6.9—
DOMINOTemperature=1, Model=Qwen3-8B, Draft size=162026.055.06—6.72—
DOMINOTemperature=1, Model=Qwen3-4B, Draft size=162026.055.04—6.45—
DFlashTemperature=1, Model=Qwen3-8B, Draft size=162026.055.04—6.73—
JetFlowBudget=32, Temperature=02026.064.86—6.48—
JetFlowBudget=16, Temperature=02026.064.77—6.24—
DFlashTemperature=1, Model=Qwen3-4B, Draft size=162026.054.76—6.38—
DFlashBudget=16, Temperature=02026.064.7—6.16—
JetFlowBudget=32, Temperature=12026.064.53—5.93—
JetFlowBudget=16, Temperature=12026.064.41—5.74—
DFlashBudget=16, Temperature=12026.064.38—5.7—
DFlashBudget=32, Temperature=02026.063.99—5.23—
DFlashBudget=32, Temperature=12026.063.74—4.81—
EAGLE-3Model=Qwen3-4B, Decoding Mode=Greedy Decoding, Temperature=02026.052.66—2.1—
DFlashModel=Qwen3-Coder-30B-A3B, Concurrency=1, Inference Engine=SGLang, Backend=FA42026.022.65696.42—
EAGLE-3Model=Qwen3-4B, Decoding Mode=Stochastic Decoding, Temperature=12026.052.38—2.08—
HyperDFlashTemperature=0, Number of drafted steps=6, Mode=Non-thinking2026.062.38—3.4—
EAGLE-3Model=Qwen3-8B, Decoding Mode=Greedy Decoding, Temperature=02026.052.29—1.91—
EAGLE-3Model=Qwen3-8B, Decoding Mode=Stochastic Decoding, Temperature=12026.052.28—1.84—
DARTTemperature=1, Model=Qwen3-8B, Draft size=602026.052.25—2.78—
DARTTemperature=0, Model=Qwen3-8B, Draft size=602026.052.24—2.78—
DARTTemperature=0, Model=Qwen3-4B, Draft size=602026.052.23—2.76—
DARTTemperature=1, Model=Qwen3-4B, Draft size=602026.052.23—2.76—
EAGLE-3Temperature=0, Model=Qwen3-4B, Draft size=602026.052.19—3.31—
HyperDFlashTemperature=0, Drafted steps per verification round=6, Decoding mode=Think-high mode2026.062.18—2.88—
HyperDFlashTemperature=1, Number of drafted steps=6, Mode=Non-thinking2026.062.17—3.16—
EAGLE-3Budget=64, Temperature=02026.062.09—3.62—
MTPTemperature=0, Drafted steps per verification round=3, Decoding mode=Think-high mode2026.062.09—2.7—
MTPTemperature=0, Number of drafted steps=3, Mode=Non-thinking2026.062.06—2.87—
EAGLE-3Temperature=1, Model=Qwen3-4B, Draft size=602026.052.05—3.19—
EAGLE-3Temperature=0, Model=Qwen3-8B, Draft size=602026.052.03—3.12—
MTPTemperature=1, Number of drafted steps=3, Mode=Non-thinking2026.062.02—2.73—
EAGLE-3Budget=64, Temperature=12026.061.99—3.49—
EAGLE-3Budget=32, Temperature=02026.061.96—3.42—
EAGLE-3Temperature=0, Model=Qwen3-4B, Draft size=162026.051.95—2.93—
HyperDFlashTemperature=1, Drafted steps per verification round=6, Decoding mode=Think-high mode2026.061.95—2.55—
EAGLE-3Budget=32, Temperature=12026.061.91—3.31—
EAGLE-3Temperature=1, Model=Qwen3-8B, Draft size=162026.051.9—2.93—
EAGLE-3Temperature=1, Model=Qwen3-8B, Draft size=602026.051.9—2.95—
MTPTemperature=1, Drafted steps per verification round=3, Decoding mode=Think-high mode2026.061.89—2.44—
EAGLE-3Temperature=1, Model=Qwen3-4B, Draft size=162026.051.87—2.83—
EAGLE-3Budget=16, Temperature=02026.061.82—3.19—
EAGLE-3Budget=16, Temperature=12026.061.81—3.09—
EAGLE-3Temperature=0, Model=Qwen3-8B, Draft size=162026.051.8—2.95—
MTPTemperature=0, Number of drafted steps=6, Mode=Non-thinking2026.061.6—3.01—
MTPTemperature=0, Drafted steps per verification round=6, Decoding mode=Think-high mode2026.061.56—2.79—
MTPTemperature=1, Number of drafted steps=6, Mode=Non-thinking2026.061.5—2.79—
Vanilla DFlashTemperature=0, Number of drafted steps=6, Mode=Non-thinking2026.061.48—1.93—
Vanilla DFlashTemperature=1, Number of drafted steps=6, Mode=Non-thinking2026.061.42—1.87—
Vanilla DFlashTemperature=0, Drafted steps per verification round=6, Decoding mode=Think-high mode2026.061.4—1.8—
MTPTemperature=1, Drafted steps per verification round=6, Decoding mode=Think-high mode2026.061.4—2.51—
Vanilla DFlashTemperature=1, Drafted steps per verification round=6, Decoding mode=Think-high mode2026.061.34—1.71—
BaselineModel=Qwen3-Coder-30B-A3B, Concurrency=1, Inference Engine=SGLang, Backend=FA42026.02—220——
DFlashtarget_model=Qwen3-4B2026.07———4.18
DFlashtarget_model=Qwen3-8B2026.07———4.39
DFlashtarget_model=Qwen3-14B2026.07———4.33
DFlashtarget_model=Gemma4-12B2026.07———3.7
DSparktarget_model=Qwen3-4B2026.07———4.86
DSparktarget_model=Qwen3-8B2026.07———5.17
DSparktarget_model=Qwen3-14B2026.07———5.02
DSparktarget_model=Gemma4-12B2026.07———4.51
Eagle3target_model=Qwen3-4B2026.07———3.77
Eagle3target_model=Qwen3-8B2026.07———4.17
Eagle3target_model=Qwen3-14B2026.07———4.01
Eagle3target_model=Gemma4-12B2026.07———4.16