# Failure Detection on ImageNet vs. WILDS New setting

Source: [Failure Detection on ImageNet vs. WILDS New setting](<https://www.sota2.com/research/sota/failure-detection-on-imagenet-vs-wilds-new-setting>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:25:43.923Z

Evaluation of model failure detection performance using the specialized 'New setting' configuration spanning ImageNet and WILDS datasets, measured by metrics like AURC, FPR95, and AUROC.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Failure Detection |
| Dataset (including any stated split) | ImageNet vs. WILDS New setting |
| Metric | AURC |
| Unit | SCALAR |
| Direction | Higher is better |
| Results loaded | 10 of 10 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [AURC](<https://www.sota2.com/research/sota/failure-detection-on-imagenet-vs-wilds-new-setting.md?metric=AURC>): Higher is better; unit: SCALAR.

- [FPR@95](<https://www.sota2.com/research/sota/failure-detection-on-imagenet-vs-wilds-new-setting.md?metric=FPR%4095>): Lower is better; unit: PERCENT.

- [AUROC](<https://www.sota2.com/research/sota/failure-detection-on-imagenet-vs-wilds-new-setting.md?metric=AUROC>): Higher is better; unit: PERCENT.

## Recorded results

| Method | AURC | Result date | Report paper |
| --- | --- | --- | --- |
| Residual | 409.89 | Not recorded | 68fa759d1e209131cea68f7a |
| Gradnorm | 382.46 | Not recorded | 68fa759d1e209131cea68f7a |
| Gradnorm | 363.03 | Not recorded | 68fa759d1e209131cea68f7a |
| Cosine | 326.97 | Not recorded | 68fa759d1e209131cea68f7a |
| MSP | 326.85 | Not recorded | 68fa759d1e209131cea68f7a |
| SIRC | 322.41 | Not recorded | 68fa759d1e209131cea68f7a |
| Energy | 318.89 | Not recorded | 68fa759d1e209131cea68f7a |
| MaxLogit | 317.46 | Not recorded | 68fa759d1e209131cea68f7a |
| Entropy | 316.26 | Not recorded | 68fa759d1e209131cea68f7a |
| TAL | 288.67 | Not recorded | 68fa759d1e209131cea68f7a |

### Residual

AURC: 409.89. Result ID: 6993d8743a6c31484918f39c.

[Reporting paper](<https://www.sota2.com/research/paper/68fa759d1e209131cea68f7a.md>)

Recorded evaluation configuration: {"Architecture":"ResNet50"}. This record does not assert independent reproduction.

### Gradnorm

AURC: 382.46. Result ID: 6993d8743a6c31484918f39b.

[Reporting paper](<https://www.sota2.com/research/paper/68fa759d1e209131cea68f7a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6afd1e209131cea2b8e2.md>)

[Implementation](<https://github.com/deeplearning-wisc/gradnorm_ood>)

Recorded evaluation configuration: {"Architecture":"ResNet50","variant":"1"}. This record does not assert independent reproduction.

### Gradnorm

AURC: 363.03. Result ID: 6993d8743a6c31484918f39d.

[Reporting paper](<https://www.sota2.com/research/paper/68fa759d1e209131cea68f7a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6afd1e209131cea2b8e2.md>)

[Implementation](<https://github.com/deeplearning-wisc/gradnorm_ood>)

Recorded evaluation configuration: {"Architecture":"ResNet50","variant":"2"}. This record does not assert independent reproduction.

### Cosine

AURC: 326.97. Result ID: 6993d8743a6c31484918f397.

[Reporting paper](<https://www.sota2.com/research/paper/68fa759d1e209131cea68f7a.md>)

Recorded evaluation configuration: {"Architecture":"ResNet50"}. This record does not assert independent reproduction.

### MSP

AURC: 326.85. Result ID: 6993d8743a6c31484918f396.

[Reporting paper](<https://www.sota2.com/research/paper/68fa759d1e209131cea68f7a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa64e41e209131cea0731a.md>)

[Implementation](<https://www.sota2.com/github.com/hendrycks/error-detection>)

Recorded evaluation configuration: {"Architecture":"ResNet50"}. This record does not assert independent reproduction.

### SIRC

AURC: 322.41. Result ID: 6993d8743a6c31484918f39e.

[Reporting paper](<https://www.sota2.com/research/paper/68fa759d1e209131cea68f7a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6eca1e209131cea42285.md>)

Recorded evaluation configuration: {"Architecture":"ResNet50"}. This record does not assert independent reproduction.

### Energy

AURC: 318.89. Result ID: 6993d8743a6c31484918f398.

[Reporting paper](<https://www.sota2.com/research/paper/68fa759d1e209131cea68f7a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa69cc1e209131cea24242.md>)

Recorded evaluation configuration: {"Architecture":"ResNet50"}. This record does not assert independent reproduction.

### MaxLogit

AURC: 317.46. Result ID: 6993d8743a6c31484918f399.

[Reporting paper](<https://www.sota2.com/research/paper/68fa759d1e209131cea68f7a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa64e41e209131cea0731a.md>)

[Implementation](<https://www.sota2.com/github.com/hendrycks/error-detection>)

Recorded evaluation configuration: {"Architecture":"ResNet50"}. This record does not assert independent reproduction.

### Entropy

AURC: 316.26. Result ID: 6993d8743a6c31484918f39a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa759d1e209131cea68f7a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6d5a1e209131cea3965a.md>)

[Implementation](<https://github.com/tianyu0207/PEBAL>)

Recorded evaluation configuration: {"Architecture":"ResNet50"}. This record does not assert independent reproduction.

### TAL

AURC: 288.67. Result ID: 6993d8743a6c31484918f39f.

[Reporting paper](<https://www.sota2.com/research/paper/68fa759d1e209131cea68f7a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa759d1e209131cea68f7a.md>)

[Implementation](<https://github.com/liuyijungoon/TAL>)

Recorded evaluation configuration: {"Architecture":"ResNet50"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.