# Failure Detection on Close Box Combined (train test)

Source: [Failure Detection on Close Box Combined (train test)](<https://www.sota2.com/research/sota/failure-detection-on-close-box-combined-train-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-13T07:48:16.068Z

Evaluation of Failure Detection performance using the train and test splits of the Close Box Combined dataset, reporting metrics like True Positive Rate (TPR), True Negative Rate (TNR), and Accuracy.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Failure Detection |
| Dataset (including any stated split) | Close Box Combined (train test) |
| Metric | TPR |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 15 of 15 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [TPR](<https://www.sota2.com/research/sota/failure-detection-on-close-box-combined-train-test.md?metric=TPR>): Higher is better; unit: PERCENT.

- [TNR](<https://www.sota2.com/research/sota/failure-detection-on-close-box-combined-train-test.md?metric=TNR>): Higher is better; unit: PERCENT.

- [Accuracy](<https://www.sota2.com/research/sota/failure-detection-on-close-box-combined-train-test.md?metric=Accuracy>): Higher is better; unit: PERCENT.

## Recorded results

| Method | TPR | Result date | Report paper |
| --- | --- | --- | --- |
| Temporal Non-Distr. Min. | 100 | Not recorded | 69b399e80636e5bf4afbc8cd |
| DDPM Loss | 100 | Not recorded | 69b399e80636e5bf4afbc8cd |
| Temporal DDPM Loss | 100 | Not recorded | 69b399e80636e5bf4afbc8cd |
| CLIP Pretrained | 100 | Not recorded | 69b399e80636e5bf4afbc8cd |
| ResNet Pretrained | 100 | Not recorded | 69b399e80636e5bf4afbc8cd |
| GPT-4o Image QA | 100 | Not recorded | 69b399e80636e5bf4afbc8cd |
| Sentinel (STAC MMD\* + GPT-4o Video QA\*) | 100 | Not recorded | 69b399e80636e5bf4afbc8cd |
| STAC For. KL | 99 | Not recorded | 69b399e80636e5bf4afbc8cd |
| STAC MMD\* | 99 | Not recorded | 69b399e80636e5bf4afbc8cd |
| Policy Encoder | 94 | Not recorded | 69b399e80636e5bf4afbc8cd |
| STAC Rev. KL | 93 | Not recorded | 69b399e80636e5bf4afbc8cd |
| Temporal Diffusion Recon. | 92 | Not recorded | 69b399e80636e5bf4afbc8cd |
| GPT-4o Video QA\* | 77 | Not recorded | 69b399e80636e5bf4afbc8cd |
| Diffusion Recon. | 37 | Not recorded | 69b399e80636e5bf4afbc8cd |
| Diffusion Output Variance | 26 | Not recorded | 69b399e80636e5bf4afbc8cd |

### Temporal Non-Distr. Min.

TPR: 100. Result ID: 69b3c07a47f34ce9249d7cc8.

[Reporting paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

Recorded evaluation configuration: {"Category":"Diffusion"}. This record does not assert independent reproduction.

### DDPM Loss

TPR: 100. Result ID: 69b3c07a47f34ce9249d7ccb.

[Reporting paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa68ef1e209131cea1f0d5.md>)

[Implementation](<https://github.com/hojonathanho/diffusion>)

Recorded evaluation configuration: {"Category":"Diffusion"}. This record does not assert independent reproduction.

### Temporal DDPM Loss

TPR: 100. Result ID: 69b3c07a47f34ce9249d7ccc.

[Reporting paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

Recorded evaluation configuration: {"Category":"Diffusion"}. This record does not assert independent reproduction.

### CLIP Pretrained

TPR: 100. Result ID: 69b3c07a47f34ce9249d7ccf.

[Reporting paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa69581e209131cea21ad0.md>)

[Implementation](<https://github.com/OpenAI/CLIP>)

Recorded evaluation configuration: {"Category":"Embed."}. This record does not assert independent reproduction.

### ResNet Pretrained

TPR: 100. Result ID: 69b3c07a47f34ce9249d7cd0.

[Reporting paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa62321e209131ce9f8328.md>)

Recorded evaluation configuration: {"Category":"Embed."}. This record does not assert independent reproduction.

### GPT-4o Image QA

TPR: 100. Result ID: 69b3c07a47f34ce9249d7cd4.

[Reporting paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa727a1e209131cea56a1e.md>)

[Implementation](<https://github.com/openai/evals>)

Recorded evaluation configuration: {"Category":"VLM"}. This record does not assert independent reproduction.

### Sentinel (STAC MMD\* + GPT-4o Video QA\*)

TPR: 100. Result ID: 69b3c07a47f34ce9249d7cd6.

[Reporting paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

Recorded evaluation configuration: {"Configuration":"Parallel Ensemble"}. This record does not assert independent reproduction.

### STAC For. KL

TPR: 99. Result ID: 69b3c07a47f34ce9249d7cd1.

[Reporting paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

Recorded evaluation configuration: {"Category":"STAC"}. This record does not assert independent reproduction.

### STAC MMD\*

TPR: 99. Result ID: 69b3c07a47f34ce9249d7cd3.

[Reporting paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

Recorded evaluation configuration: {"Category":"STAC"}. This record does not assert independent reproduction.

### Policy Encoder

TPR: 94. Result ID: 69b3c07a47f34ce9249d7cce.

[Reporting paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa64f91e209131cea077ac.md>)

[Implementation](<https://github.com/pokaxpoka/deep_Mahalanobis_detector>)

Recorded evaluation configuration: {"Category":"Embed."}. This record does not assert independent reproduction.

### STAC Rev. KL

TPR: 93. Result ID: 69b3c07a47f34ce9249d7cd2.

[Reporting paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

Recorded evaluation configuration: {"Category":"STAC"}. This record does not assert independent reproduction.

### Temporal Diffusion Recon.

TPR: 92. Result ID: 69b3c07a47f34ce9249d7cca.

[Reporting paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

Recorded evaluation configuration: {"Category":"Diffusion"}. This record does not assert independent reproduction.

### GPT-4o Video QA\*

TPR: 77. Result ID: 69b3c07a47f34ce9249d7cd5.

[Reporting paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

Recorded evaluation configuration: {"Category":"VLM"}. This record does not assert independent reproduction.

### Diffusion Recon.

TPR: 37. Result ID: 69b3c07a47f34ce9249d7cc9.

[Reporting paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f271e209131cea44174.md>)

Recorded evaluation configuration: {"Category":"Diffusion"}. This record does not assert independent reproduction.

### Diffusion Output Variance

TPR: 26. Result ID: 69b3c07a47f34ce9249d7ccd.

[Reporting paper](<https://www.sota2.com/research/paper/69b399e80636e5bf4afbc8cd.md>)

Recorded evaluation configuration: {"Category":"Diffusion"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
