Code on MBPP (Speedup/Tau)
7.68SpeedupBASTION
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| BASTIONModel=Qwen3-4B, Temperature=1, Hardware=H1002026.05 | 7.68 | 8.74 | |
| BASTIONModel=Qwen3-4B, Temperature=0, Hardware=H1002026.05 | 7.22 | 8.81 | |
| BASTIONModel=Qwen3-8B, Temperature=0, Hardware=H1002026.05 | 7.04 | 8.62 | |
| BASTIONModel=Qwen3-8B, Temperature=1, Hardware=H1002026.05 | 6.66 | 8.57 | |
| DFlashModel=Qwen3-4B, Temperature=0, Hardware=H100, Block size=162026.05 | 5.4 | 6.12 | |
| DFlashModel=Qwen3-8B, Temperature=0, Hardware=H100, Block size=162026.05 | 5.27 | 5.98 | |
| DFlashModel=Qwen3-4B, Temperature=1, Hardware=H100, Block size=162026.05 | 4.99 | 5.63 | |
| DFlashModel=Qwen3-8B, Temperature=1, Hardware=H100, Block size=162026.05 | 4.6 | 5.26 | |
| EAGLE-3Model=Qwen3-4B, Temperature=0, Hardware=H100, Tree budget size=602026.05 | 2.95 | 2.47 | |
| EAGLE-3Model=Qwen3-4B, Temperature=1, Hardware=H100, Tree budget size=602026.05 | 2.94 | 2.38 | |
| EAGLE-3Model=Qwen3-8B, Temperature=0, Hardware=H100, Tree budget size=602026.05 | 2.82 | 2.2 | |
| EAGLE-3Model=Qwen3-8B, Temperature=1, Hardware=H100, Tree budget size=602026.05 | 2.58 | 2.21 |