Aggregate Performance on Overall Average
4.12SpeedupBlockPilot
Evaluation results
| Method | Links | ||
|---|---|---|---|
| BlockPilotModel=Qwen2.5-Coder-32B-Instruct, Temperature=02026.06 | 4.12 | 6.05 | |
| BlockPilotModel=Qwen2.5-Coder-32B-Instruct, Temperature=12026.06 | 3.95 | 5.84 | |
| BlockPilotModel=Llama-3.1-8B-Instruct, Temperature=02026.06 | 3.25 | 4.82 | |
| BlockPilotModel=Llama-3.1-8B-Instruct, Temperature=12026.06 | 2.4 | 3.68 |