Robotic Program Synthesis on ROBOEVAL
5Python Error RateQwen2.5-Coder
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Qwen2.5-CoderModel Parameters=7B, Variant=base2026.05 | 5 | 19 | 26 | 30 | |
| Qwen2.5-CoderModel Parameters=7B, Variant=ROBO-INSTRUCT2026.05 | 9 | 16 | 24 | 31 | |
| Qwen2.5-CoderModel Parameters=1.5B, Variant=ours2026.05 | 11 | 28 | 27 | 14 | |
| Qwen2.5-CoderModel Parameters=1.5B, Variant=base2026.05 | 77 | 3 | 0 | 0 |