# Image Classification on ColoredMNIST biased scenario (test)

Source: [Image Classification on ColoredMNIST biased scenario (test)](<https://www.sota2.com/research/sota/image-classification-on-coloredmnist-biased-scenario-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:35:13.634Z

Evaluation of Image Classification performance on the biased scenario test split of the Colored MNIST dataset.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Image Classification |
| Dataset (including any stated split) | ColoredMNIST biased scenario (test) |
| Metric | Average Accuracy |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 7 of 7 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Average Accuracy](<https://www.sota2.com/research/sota/image-classification-on-coloredmnist-biased-scenario-test.md?metric=Average%20Accuracy>): Higher is better; unit: PERCENT.

- [Worst-Group Accuracy](<https://www.sota2.com/research/sota/image-classification-on-coloredmnist-biased-scenario-test.md?metric=Worst-Group%20Accuracy>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Average Accuracy | Result date | Report paper |
| --- | --- | --- | --- |
| DeYO | 78.24 | Not recorded | 68fa72801e209131cea56d2d |
| MEMO | 63.77 | Not recorded | 68fa72801e209131cea56d2d |
| ResNet-18-BN | 63.4 | Not recorded | 68fa72801e209131cea56d2d |
| SENTRY | 63.23 | Not recorded | 68fa72801e209131cea56d2d |
| EATA | 60.81 | Not recorded | 68fa72801e209131cea56d2d |
| SAR | 58.37 | Not recorded | 68fa72801e209131cea56d2d |
| Tent | 57.06 | Not recorded | 68fa72801e209131cea56d2d |

### DeYO

Average Accuracy: 78.24. Result ID: 69922f423a6c3148490d240a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa72801e209131cea56d2d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72801e209131cea56d2d.md>)

[Implementation](<https://whitesnowdrop.github.io/DeYO/>)

Recorded evaluation configuration: {"Scenario":"Biased","Backbone":"ResNet-18-BN"}. This record does not assert independent reproduction.

### MEMO

Average Accuracy: 63.77. Result ID: 69922f423a6c3148490d2406.

[Reporting paper](<https://www.sota2.com/research/paper/68fa72801e209131cea56d2d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6da51e209131cea3b236.md>)

Recorded evaluation configuration: {"Scenario":"Biased","Backbone":"ResNet-18-BN"}. This record does not assert independent reproduction.

### ResNet-18-BN

Average Accuracy: 63.4. Result ID: 69922f423a6c3148490d2404.

[Reporting paper](<https://www.sota2.com/research/paper/68fa72801e209131cea56d2d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa62321e209131ce9f8328.md>)

Recorded evaluation configuration: {"Scenario":"Biased","Backbone":"ResNet-18-BN"}. This record does not assert independent reproduction.

### SENTRY

Average Accuracy: 63.23. Result ID: 69922f423a6c3148490d2407.

[Reporting paper](<https://www.sota2.com/research/paper/68fa72801e209131cea56d2d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6afc1e209131cea2b86e.md>)

[Implementation](<https://github.com/virajprabhu/SENTRY>)

Recorded evaluation configuration: {"Scenario":"Biased","Backbone":"ResNet-18-BN"}. This record does not assert independent reproduction.

### EATA

Average Accuracy: 60.81. Result ID: 69922f423a6c3148490d2408.

[Reporting paper](<https://www.sota2.com/research/paper/68fa72801e209131cea56d2d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6ca11e209131cea35123.md>)

Recorded evaluation configuration: {"Scenario":"Biased","Backbone":"ResNet-18-BN"}. This record does not assert independent reproduction.

### SAR

Average Accuracy: 58.37. Result ID: 69922f423a6c3148490d2409.

[Reporting paper](<https://www.sota2.com/research/paper/68fa72801e209131cea56d2d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6e961e209131cea40e77.md>)

Recorded evaluation configuration: {"Scenario":"Biased","Backbone":"ResNet-18-BN"}. This record does not assert independent reproduction.

### Tent

Average Accuracy: 57.06. Result ID: 69922f423a6c3148490d2405.

[Reporting paper](<https://www.sota2.com/research/paper/68fa72801e209131cea56d2d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa69941e209131cea22d2b.md>)

Recorded evaluation configuration: {"Scenario":"Biased","Backbone":"ResNet-18-BN"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.