Repository-level patch generation on SWE-Bench Lite (held-out 25% eval)
102.7Utility Difference (Δπ)Rfx
Evaluation Results
| Method | Links | |
|---|---|---|
| RfxGenerator=gpt-5-mini2026.06 | 102.7 | |
| bayesian_greedyGenerator=gpt-5-mini2026.06 | 62 | |
| bayesian_DPGenerator=gpt-5-mini2026.06 | 62 | |
| SRGenerator=gpt-5-mini2026.06 | 24.4 | |
| tCsynGenerator=gpt-5-mini2026.06 | 21.7 | |
| BoNGenerator=gpt-5-mini2026.06 | 17.3 | |
| tCllmGenerator=gpt-5-mini2026.06 | 13.7 | |
| fixed_pipelineGenerator=gpt-5-mini2026.06 | 11.6 |