Failure Detection on Close Box Out-of-Distribution
100TPRTemporal Non-Distr. Min.
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Temporal Non-Distr. Min.Failure Detector Category=Diffusion2026.03 | 100 | 27 | 12.35 | |
| DDPM Loss (Eq. A.1)Failure Detector Category=Diffusion2026.03 | 100 | 94 | 14.54 | |
| Temporal DDPM LossFailure Detector Category=Diffusion2026.03 | 100 | 37 | 13.66 | |
| Policy EncoderFailure Detector Category=Embed.2026.03 | 100 | 0 | 1.59 | |
| CLIP PretrainedFailure Detector Category=Embed.2026.03 | 100 | 0 | 8.2 | |
| ResNet PretrainedFailure Detector Category=Embed.2026.03 | 100 | 0 | 15.51 | |
| GPT-4o Image QAFailure Detector Category=VLM2026.03 | 100 | 0 | 23.2 | |
| Gemini 1.5 Pro Image QAFailure Detector Category=VLM2026.03 | 100 | 0 | 23.2 | |
| Gemini 1.5 Pro Video QAFailure Detector Category=VLM2026.03 | 100 | 50 | 20.2 | |
| SentinelFailure Detector Category=Combined, Configuration=STAC MMD* + GPT-4o Video QA*2026.03 | 100 | 90 | 14.25 | |
| STAC For. KLFailure Detector Category=STAC2026.03 | 99 | 85 | 14.04 | |
| STAC MMD*Failure Detector Category=STAC2026.03 | 99 | 93 | 14.72 | |
| STAC Rev. KLFailure Detector Category=STAC2026.03 | 93 | 97 | 15.12 | |
| Temporal Diffusion Recon.Failure Detector Category=Diffusion2026.03 | 92 | 100 | 15.75 | |
| GPT-4o Video QA*Failure Detector Category=VLM2026.03 | 69 | 95 | 21.02 | |
| Claude 3.5 Sonnet Image QAFailure Detector Category=VLM2026.03 | 69 | 10 | 23.2 | |
| Claude 3.5 Sonnet Video QAFailure Detector Category=VLM2026.03 | 44 | 40 | 23.2 | |
| Diffusion Recon.Failure Detector Category=Diffusion2026.03 | 40 | 100 | 17.08 | |
| Diffusion Output VarianceFailure Detector Category=Diffusion2026.03 | 28 | 100 | 17.27 |