Task Progression Failure Detection on Cover Object Policy Success Rate (In-Distribution)
100TPRTemporal Non-Distr. Min.
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Temporal Non-Distr. Min.2026.03 | 100 | 93 | 2.4 | |
| STAC For. KLvariant=Forward KL2026.03 | 100 | 95 | 2.4 | |
| STAC Rev. KLvariant=Reverse KL2026.03 | 100 | 93 | 2.4 | |
| STAC MMD*variant=MMD2026.03 | 100 | 93 | 2.4 | |
| GPT-4o Image QAmodality=Image QA2026.03 | 100 | 7 | 12 | |
| GPT-4o Video QAmodality=Video QA2026.03 | 100 | 5 | 5.6 | |
| Gemini 1.5 Pro Image QAmodality=Image QA2026.03 | 100 | 5 | 12 | |
| Gemini 1.5 Pro Video QAmodality=Video QA2026.03 | 100 | 0 | 5.6 | |
| Claude 3.5 Sonnet Image QAmodality=Image QA2026.03 | 100 | 81 | 12 | |
| Claude 3.5 Sonnet Video QAmodality=Video QA2026.03 | 100 | 84 | 12 | |
| Claude 3.5 Sonnet Video QA + Success Videomodality=Video QA, context=Success Video2026.03 | 100 | 77 | 12 | |
| Claude 3.5 Sonnet Video QA + Goal Imagesmodality=Video QA, context=Goal Images2026.03 | 100 | 93 | 5.6 | |
| Claude 3.5 Sonnet Prompt Ensemble*strategy=Prompt Ensemble2026.03 | 100 | 93 | 12 | |
| Sentinel (STAC MMD* + Claude 3.5 Sonnet Video QA)components=STAC MMD* + Claude 3.5 Sonnet Video QA2026.03 | 100 | 79 | 2.4 | |
| Sentinel (STAC MMD* + Claude 3.5 Sonnet Prompt Ensemble*)components=STAC MMD* + Claude 3.5 Sonnet Prompt Ensemble*2026.03 | 100 | 88 | 2.4 | |
| Diffusion Output Variance2026.03 | 0 | 77 | — |