List your company
ResearchFormerly WizWand

Code on MBPP (Speedup/Tau)

7.68SpeedupBASTION
2.3763.7535.136.507May 28, 2026
Updated 5mo ago

Evaluation results

MethodLinks
BASTIONModel=Qwen3-4B, Temperature=1, Hardware=H1002026.057.688.74
BASTIONModel=Qwen3-4B, Temperature=0, Hardware=H1002026.057.228.81
BASTIONModel=Qwen3-8B, Temperature=0, Hardware=H1002026.057.048.62
BASTIONModel=Qwen3-8B, Temperature=1, Hardware=H1002026.056.668.57
DFlashModel=Qwen3-4B, Temperature=0, Hardware=H100, Block size=162026.055.46.12
DFlashModel=Qwen3-8B, Temperature=0, Hardware=H100, Block size=162026.055.275.98
DFlashModel=Qwen3-4B, Temperature=1, Hardware=H100, Block size=162026.054.995.63
DFlashModel=Qwen3-8B, Temperature=1, Hardware=H100, Block size=162026.054.65.26
EAGLE-3Model=Qwen3-4B, Temperature=0, Hardware=H100, Tree budget size=602026.052.952.47
EAGLE-3Model=Qwen3-4B, Temperature=1, Hardware=H100, Tree budget size=602026.052.942.38
EAGLE-3Model=Qwen3-8B, Temperature=0, Hardware=H100, Tree budget size=602026.052.822.2
EAGLE-3Model=Qwen3-8B, Temperature=1, Hardware=H100, Tree budget size=602026.052.582.21