Code Generation on SWE-Bench (Efficiency Metrics)
2.96Speedup RatioBlockPilot
Evaluation results
| Method | Links | ||
|---|---|---|---|
| BlockPilotModel=Qwen3-8B, Temperature=02026.06 | 2.96 | — | |
| DFlashModel=Qwen3-8B, Temperature=0, Block size (n)=162026.06 | 2.72 | — | |
| BlockPilotModel=Qwen3-4B, Temperature=12026.06 | 2.58 | — | |
| DFlashModel=Qwen3-8B, Temperature=0, Block size (n)=322026.06 | 2.5 | — | |
| BlockPilotModel=Qwen3-4B, Temperature=02026.06 | 2.45 | — | |
| DFlashModel=Qwen3-8B, Temperature=0, Block size (n)=82026.06 | 2.45 | — | |
| DFlashModel=Qwen3-4B, Temperature=0, Block size (n)=162026.06 | 2.33 | — | |
| DFlashModel=Qwen3-4B, Temperature=1, Block size (n)=162026.06 | 2.33 | — | |
| BlockPilotModel=Qwen3-8B, Temperature=12026.06 | 2.3 | — | |
| DFlashModel=Qwen3-4B, Temperature=1, Block size (n)=82026.06 | 2.17 | — | |
| DFlashModel=Qwen3-8B, Temperature=1, Block size (n)=162026.06 | 2.09 | — | |
| DFlashModel=Qwen3-8B, Temperature=1, Block size (n)=322026.06 | 2.01 | — | |
| DFlashModel=Qwen3-8B, Temperature=0, Block size (n)=42026.06 | 1.96 | — | |
| DFlashModel=Qwen3-4B, Temperature=0, Block size (n)=82026.06 | 1.95 | — | |
| DFlashModel=Qwen3-8B, Temperature=1, Block size (n)=82026.06 | 1.94 | — | |
| DFlashModel=Qwen3-4B, Temperature=1, Block size (n)=322026.06 | 1.91 | — | |
| DFlashModel=Qwen3-4B, Temperature=1, Block size (n)=42026.06 | 1.79 | — | |
| DFlashModel=Qwen3-4B, Temperature=0, Block size (n)=322026.06 | 1.69 | — | |
| DFlashModel=Qwen3-8B, Temperature=1, Block size (n)=42026.06 | 1.66 | — | |
| DFlashModel=Qwen3-4B, Temperature=0, Block size (n)=42026.06 | 1.59 | — | |
| EAGLE-3Model=Qwen3-8B, Temperature=02026.06 | 1.47 | — | |
| EAGLE-3Model=Qwen3-4B, Temperature=12026.06 | 1.47 | — | |
| EAGLE-3Model=Qwen3-4B, Temperature=02026.06 | 1.39 | — | |
| EAGLE-3Model=Qwen3-8B, Temperature=12026.06 | 1.33 | — |