# Failure Detection on Close Box In-Distribution (train test)

Source: [Failure Detection on Close Box In-Distribution (train test)](<https://www.sota2.com/research/sota/failure-detection-on-close-box-in-distribution-train-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-13T07:48:16.012Z

Evaluation of Failure Detection performance using the train and test splits of the Close Box In-Distribution dataset, reporting metrics like True Positive Rate (TPR), True Negative Rate (TNR), and detection time.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Failure Detection |
| Dataset (including any stated split) | Close Box In-Distribution (train test) |
| Metric | TPR |
| Unit | SCALAR |
| Direction | Higher is better |
| Results loaded | 15 of 15 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [TPR](<https://www.sota2.com/research/sota/failure-detection-on-close-box-in-distribution-train-test.md?metric=TPR>): Higher is better; unit: SCALAR.

- [TNR](<https://www.sota2.com/research/sota/failure-detection-on-close-box-in-distribution-train-test.md?metric=TNR>): Higher is better; unit: SCALAR.

- [Detection Time (s)](<https://www.sota2.com/research/sota/failure-detection-on-close-box-in-distribution-train-test.md?metric=Detection%20Time%20(s)>): Lower is better; unit: TIME.

## Recorded results

| Method | TPR | Result date | Report paper |
| --- | --- | --- | --- |
| Temporal Non-Distr. Min. | 1 | Not recorded | 69b399e80636e5bf4afbc8cd |
| Temporal Diffusion Recon. | 1 | Not recorded | 69b399e80636e5bf4afbc8cd |
| DDPM Loss | 1 | Not recorded | 69b399e80636e5bf4afbc8cd |
| Temporal DDPM Loss | 1 | Not recorded | 69b399e80636e5bf4afbc8cd |
| CLIP Pretrained | 1 | Not recorded | 69b399e80636e5bf4afbc8cd |
| ResNet Pretrained | 1 | Not recorded | 69b399e80636e5bf4afbc8cd |
| STAC For. KL | 1 | Not recorded | 69b399e80636e5bf4afbc8cd |
| STAC Rev. KL | 1 | Not recorded | 69b399e80636e5bf4afbc8cd |
| STAC MMD\* | 1 | Not recorded | 69b399e80636e5bf4afbc8cd |
| GPT-4o Image QA | 1 | Not recorded | 69b399e80636e5bf4afbc8cd |
| GPT-4o Video QA\* | 1 | Not recorded | 69b399e80636e5bf4afbc8cd |
| Sentinel (STAC MMD\* + GPT-4o Video QA\*) | 1 | Not recorded | 69b399e80636e5bf4afbc8cd |
| Diffusion Recon. | 0.33 | Not recorded | 69b399e80636e5bf4afbc8cd |
| Diffusion Output Variance | 0.33 | Not recorded | 69b399e80636e5bf4afbc8cd |
| Policy Encoder | 0.25 | Not recorded | 69b399e80636e5bf4afbc8cd |

### Temporal Non-Distr. Min.

TPR: 1. Result ID: 69b3c07a47f34ce9249d7caa.

[Reporting paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

Recorded evaluation configuration: {"Category":"Diffusion"}. This record does not assert independent reproduction.

### Temporal Diffusion Recon.

TPR: 1. Result ID: 69b3c07a47f34ce9249d7cac.

[Reporting paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

Recorded evaluation configuration: {"Category":"Diffusion"}. This record does not assert independent reproduction.

### DDPM Loss

TPR: 1. Result ID: 69b3c07a47f34ce9249d7cad.

[Reporting paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa68ef1e209131cea1f0d5.md>)

[Implementation](<https://github.com/hojonathanho/diffusion>)

Recorded evaluation configuration: {"Category":"Diffusion"}. This record does not assert independent reproduction.

### Temporal DDPM Loss

TPR: 1. Result ID: 69b3c07a47f34ce9249d7cae.

[Reporting paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

Recorded evaluation configuration: {"Category":"Diffusion"}. This record does not assert independent reproduction.

### CLIP Pretrained

TPR: 1. Result ID: 69b3c07a47f34ce9249d7cb1.

[Reporting paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa69581e209131cea21ad0.md>)

[Implementation](<https://github.com/OpenAI/CLIP>)

Recorded evaluation configuration: {"Category":"Embed."}. This record does not assert independent reproduction.

### ResNet Pretrained

TPR: 1. Result ID: 69b3c07a47f34ce9249d7cb2.

[Reporting paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa62321e209131ce9f8328.md>)

Recorded evaluation configuration: {"Category":"Embed."}. This record does not assert independent reproduction.

### STAC For. KL

TPR: 1. Result ID: 69b3c07a47f34ce9249d7cb3.

[Reporting paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

Recorded evaluation configuration: {"Category":"STAC"}. This record does not assert independent reproduction.

### STAC Rev. KL

TPR: 1. Result ID: 69b3c07a47f34ce9249d7cb4.

[Reporting paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

Recorded evaluation configuration: {"Category":"STAC"}. This record does not assert independent reproduction.

### STAC MMD\*

TPR: 1. Result ID: 69b3c07a47f34ce9249d7cb5.

[Reporting paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

Recorded evaluation configuration: {"Category":"STAC"}. This record does not assert independent reproduction.

### GPT-4o Image QA

TPR: 1. Result ID: 69b3c07a47f34ce9249d7cb6.

[Reporting paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa727a1e209131cea56a1e.md>)

[Implementation](<https://github.com/openai/evals>)

Recorded evaluation configuration: {"Category":"VLM"}. This record does not assert independent reproduction.

### GPT-4o Video QA\*

TPR: 1. Result ID: 69b3c07a47f34ce9249d7cb7.

[Reporting paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

Recorded evaluation configuration: {"Category":"VLM"}. This record does not assert independent reproduction.

### Sentinel (STAC MMD\* + GPT-4o Video QA\*)

TPR: 1. Result ID: 69b3c07a47f34ce9249d7cb8.

[Reporting paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

Recorded evaluation configuration: {"Configuration":"Parallel Ensemble"}. This record does not assert independent reproduction.

### Diffusion Recon.

TPR: 0.33. Result ID: 69b3c07a47f34ce9249d7cab.

[Reporting paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f271e209131cea44174.md>)

Recorded evaluation configuration: {"Category":"Diffusion"}. This record does not assert independent reproduction.

### Diffusion Output Variance

TPR: 0.33. Result ID: 69b3c07a47f34ce9249d7caf.

[Reporting paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

Recorded evaluation configuration: {"Category":"Diffusion"}. This record does not assert independent reproduction.

### Policy Encoder

TPR: 0.25. Result ID: 69b3c07a47f34ce9249d7cb0.

[Reporting paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa64f91e209131cea077ac.md>)

[Implementation](<https://github.com/pokaxpoka/deep_Mahalanobis_detector>)

Recorded evaluation configuration: {"Category":"Embed."}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
