Task Generation on SWE OOD (held-out repositories)
19.59UtilityPROPEL (OOD)
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| PROPEL (OOD)Solver=27B2026.06 | 19.59 | 75.13 | 89.8 | 9.3 | 71.8 | |
| Base (OOD)Solver=27B2026.06 | 9.84 | 83.56 | 81.1 | 8.5 | 60.5 |
Loading the SOTA2 catalog…
| Method | Links | |||||
|---|---|---|---|---|---|---|
| PROPEL (OOD)Solver=27B2026.06 | 19.59 | 75.13 | 89.8 | 9.3 | 71.8 | |
| Base (OOD)Solver=27B2026.06 | 9.84 | 83.56 | 81.1 | 8.5 | 60.5 |