False Success Detection on AppWorld task-disjoint
53.7AUROC (Judge)GPT-4o
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| GPT-4oDomain=Pers. apps, Agent=Coding agent2026.06 | 53.7 | — | |
| TF-IDF + XGBoostDomain=Pers. apps, Agent=Coding agent2026.06 | — | 95.3 |
Loading the SOTA2 catalog…