Colored MNIST
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
Colored-MNIST (val)
99.95Unbiased Accuracy
4
Colored MNIST Unbiased 33/33/33
95Specificity
2
Colored MNIST red-biased family Severity IV 60 20 20
86Recall
2
Colored MNIST red-biased Severity III 80/10/10
100Recall
2
Colored MNIST red-biased Severity II 90/5/5
100Recall
2
Colored MNIST red-biased family Severity I 99/.5/.5
100Recall
2
Colored MNIST In-distribution vs Textures protocol (test)
100TNR @ 95% TPR
2
Colored MNIST In-distribution vs iSUN protocol (test)
100TNR @ 95% TPR
2
Colored MNIST In-distribution vs LSUN protocol (test)
1TNR @ 95% TPR
2
Colored MNIST In-distribution vs Spurious OOD protocol (test)
0.958TNR @ 95% TPR
2