Failure Detection on Close Box In-Distribution
100TPRTemporal Non-Distr. Min.
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Temporal Non-Distr. Min.Failure Detector Category=Diffusion2026.03 | 100 | 97 | 5 | |
| Temporal Diffusion Recon.Failure Detector Category=Diffusion2026.03 | 100 | 96 | 8.47 | |
| DDPM Loss (Eq. A.1)Failure Detector Category=Diffusion2026.03 | 100 | 90 | 8.27 | |
| Temporal DDPM LossFailure Detector Category=Diffusion2026.03 | 100 | 95 | 7.53 | |
| CLIP PretrainedFailure Detector Category=Embed.2026.03 | 100 | 95 | 15.73 | |
| ResNet PretrainedFailure Detector Category=Embed.2026.03 | 100 | 95 | 17.87 | |
| STAC For. KLFailure Detector Category=STAC2026.03 | 100 | 90 | 6.6 | |
| STAC Rev. KLFailure Detector Category=STAC2026.03 | 100 | 95 | 7.6 | |
| STAC MMD*Failure Detector Category=STAC2026.03 | 100 | 94 | 7.2 | |
| GPT-4o Image QAFailure Detector Category=VLM2026.03 | 100 | 0 | 23.2 | |
| GPT-4o Video QA*Failure Detector Category=VLM2026.03 | 100 | 89 | 21.2 | |
| Gemini 1.5 Pro Image QAFailure Detector Category=VLM2026.03 | 100 | 0 | 21.2 | |
| Gemini 1.5 Pro Video QAFailure Detector Category=VLM2026.03 | 100 | 57 | 17.2 | |
| Claude 3.5 Sonnet Image QAFailure Detector Category=VLM2026.03 | 100 | 6 | 23.2 | |
| SentinelFailure Detector Category=Combined, Configuration=STAC MMD* + GPT-4o Video QA*2026.03 | 100 | 86 | 5.47 | |
| Claude 3.5 Sonnet Video QAFailure Detector Category=VLM2026.03 | 83 | 31 | 23.2 | |
| Diffusion Recon.Failure Detector Category=Diffusion2026.03 | 33 | 95 | 13.6 | |
| Diffusion Output VarianceFailure Detector Category=Diffusion2026.03 | 33 | 94 | 14 | |
| Policy EncoderFailure Detector Category=Embed.2026.03 | 25 | 98 | 16.27 |