Failure prediction stopping on AppWorld
78.8Max Accuracyα-STOP
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| α-STOPPredictor=attention-based failure predictor2026.06 | 78.8 | 74.4 | 0.0002 | |
| Plug-in ThresholdPredictor=attention-based failure predictor2026.06 | 76.8 | 71.2 | 0.0074 | |
| FIRMBOUNDPredictor=attention-based failure predictor2026.06 | 75.8 | 70.1 | 0.0129 | |
| EndtoEndRLPredictor=attention-based failure predictor2026.06 | 73.9 | 69.4 | 0.0125 | |
| ALERT*Predictor=attention-based failure predictor2026.06 | 73.2 | 68 | 0.0198 | |
| LLM Judge (best)Predictor=attention-based failure predictor2026.06 | 70.9 | — | — |