Bug Fixing on CodeAssistBench (test)
54.1AccuracyGPT-5
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| GPT-5Interface=CodeStruct2026.04 | 54.1 | 122.1 | 1.18 | 550 | 16.74 | |
| GPT-5Interface=Baseline2026.04 | 53.3 | 143.9 | 1.27 | 566 | 19.57 | |
| GPT-5-miniInterface=CodeStruct2026.04 | 51.9 | 83.1 | 0.89 | 630 | 2.3 | |
| GPT-5-miniInterface=Baseline2026.04 | 51.1 | 125.3 | 1.26 | 648 | 3.45 | |
| GPT-5-nanoInterface=CodeStruct2026.04 | 48.1 | 52.8 | 1.05 | 652 | 0.32 | |
| GPT-5-nanoInterface=Baseline2026.04 | 46.7 | 56 | 1.25 | 742 | 0.34 | |
| Qwen3-CoderInterface=CodeStruct2026.04 | 31.9 | 779.2 | 7 | 826 | 363.24 | |
| Qwen3-CoderInterface=Baseline2026.04 | 31.1 | 829.1 | 6.95 | 796 | 385.61 | |
| Qwen3-32BInterface=CodeStruct2026.04 | 20 | 142.4 | 1.25 | 776 | 38.71 | |
| Qwen3-32BInterface=Baseline2026.04 | 15.6 | 110.5 | 2.18 | 858 | 29.01 | |
| Qwen3-8BInterface=CodeStruct2026.04 | 14.1 | 0.55 | 0.17 | 944 | 0.04 | |
| Qwen3-8BInterface=Baseline2026.04 | 13.3 | 0.63 | 0.21 | 984 | 0.05 |