Failure prediction on SWE-agent (Transfer held-out)
0.69ROC-AUCSrc-aware
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Src-aware2026.06 | 0.69 | — | — | |
| flat plus dependency layer (f+d)Features=f+d2026.06 | 0.69 | 0.623 | 0.001 | |
| size-normalized dependency layer (dep)Features=dep2026.06 | 0.646 | 0.573 | 0.001 | |
| GIN2026.06 | 0.633 | — | — | |
| HGT2026.06 | 0.633 | — | — | |
| run size (flat)Features=flat2026.06 | 0.624 | 0.559 | 0.001 | |
| R-GCN2026.06 | 0.541 | — | — |