Task Progression Failure Detection on Cover Object Policy Success Rate: 3% (Out-of-Distribution)
100TPRGPT-4o Image QA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| GPT-4o Image QAmodality=Image QA2026.03 | 100 | 0 | 11.03 | |
| GPT-4o Video QAmodality=Video QA2026.03 | 100 | 0 | 10.06 | |
| Gemini 1.5 Pro Video QAmodality=Video QA2026.03 | 100 | 0 | 7.54 | |
| Claude 3.5 Sonnet Video QA + Success Videomodality=Video QA, context=Success Video2026.03 | 94 | 100 | 11.59 | |
| Gemini 1.5 Pro Image QAmodality=Image QA2026.03 | 91 | 0 | 11.15 | |
| Sentinel (STAC MMD* + Claude 3.5 Sonnet Prompt Ensemble*)components=STAC MMD* + Claude 3.5 Sonnet Prompt Ensemble*2026.03 | 88 | 100 | 8.69 | |
| Claude 3.5 Sonnet Prompt Ensemble*strategy=Prompt Ensemble2026.03 | 85 | 100 | 12 | |
| Sentinel (STAC MMD* + Claude 3.5 Sonnet Video QA)components=STAC MMD* + Claude 3.5 Sonnet Video QA2026.03 | 82 | 100 | 8.68 | |
| Claude 3.5 Sonnet Video QAmodality=Video QA2026.03 | 79 | 100 | 12 | |
| Claude 3.5 Sonnet Video QA + Goal Imagesmodality=Video QA, context=Goal Images2026.03 | 76 | 100 | 11.74 | |
| Claude 3.5 Sonnet Image QAmodality=Image QA2026.03 | 70 | 100 | 12 | |
| STAC MMD*variant=MMD2026.03 | 9 | 100 | 7.73 | |
| Temporal Non-Distr. Min.2026.03 | 6 | 100 | 7.6 | |
| STAC For. KLvariant=Forward KL2026.03 | 3 | 100 | 6.4 | |
| STAC Rev. KLvariant=Reverse KL2026.03 | 3 | 100 | 6.4 | |
| Diffusion Output Variance2026.03 | 0 | 100 | — |