Bug Fixing on SWE-Bench SWE-Agent-style workflows Verified
67.2Pass@1GPT-5
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| GPT-5Interface=CodeStruct2026.04 | 67.2 | 366.3 | 0.44 | 16,307 | 462.2 | |
| Qwen3-CoderInterface=CodeStruct2026.04 | 66.2 | 705.3 | 2.17 | 32,346 | 321.3 | |
| GPT-5Interface=Baseline2026.04 | 66 | 452.7 | 0.81 | 16,436 | 574 | |
| GPT-5-miniInterface=CodeStruct2026.04 | 62 | 404.5 | 0.35 | 14,811 | 101.8 | |
| Qwen3-CoderInterface=Baseline2026.04 | 61.2 | 805.8 | 1.5 | 26,961 | 365.3 | |
| GPT-5-miniInterface=Baseline2026.04 | 60.4 | 593.7 | 1.27 | 18,560 | 151 | |
| GPT-5-nanoInterface=CodeStruct2026.04 | 40.4 | 1,137.4 | 0.95 | 27,278 | 57.3 | |
| GPT-5-nanoInterface=Baseline2026.04 | 19.6 | 808 | 0.86 | 24,037 | 40.7 | |
| Qwen3-32BInterface=CodeStruct2026.04 | 16 | 302.1 | 1.03 | 24,653 | 45.9 | |
| Qwen3-32BInterface=Baseline2026.04 | 14.8 | 366 | 1.23 | 25,543 | 55.6 | |
| Qwen3-8BInterface=Baseline2026.04 | 13.2 | 84 | 0.11 | 8,833 | 2.36 | |
| Qwen3-8BInterface=CodeStruct2026.04 | 13 | 51.8 | 0.08 | 8,313 | 1.46 |