Sequence-level risk prediction on SWE-smith (test)
67.6AUCGRU on tool sequence
Evaluation Results
| Method | Links | |
|---|---|---|
| GRU on tool sequenceTraining=yes, Live=no, Interpretable=no2026.06 | 67.6 | |
| Bag-of-tools LRTraining=yes, Live=no, Interpretable=partial2026.06 | 65.8 | |
| Trace-risk modelTraining=scalar only, Live=yes, Interpretable=yes2026.06 | 63.7 | |
| R(τ) onlyTraining=no, Live=yes, Interpretable=yes2026.06 | 54.8 | |
| Mean action risk onlyTraining=no, Live=yes, Interpretable=yes2026.06 | 53.9 | |
| Random2026.06 | 50 |