CMNIST
Benchmarks
TaskDatasetSOTA resultTrendResults
CMNIST-75sp unbiased (test)
82.73Accuracy
60
CMNIST (test)
93.54Test Accuracy
55
cMNIST (test)
85.26WGA
33
cMNIST
75.75WGA
26
CMNIST 0.5% bias ratio unbiased (test)
97.52Accuracy
17
CMNIST color
36.37Accuracy
17
CMNIST-sp Superpixel distribution shift
68.94Accuracy
15
CMNIST color OOD
48.1Accuracy
14
CMNIST color ID
79.3Accuracy
14
CMNIST color GOOD
53.3Accuracy
14
cMNIST (test)
83.37Worst Group Accuracy (WGA)
13
CMNIST GraphOOD (test)
52.17Accuracy
13
CMNIST
28.49Accuracy
13
CMNIST
69.66AA
12
CMNIST-SP
62.13Accuracy
12
CMNIST 5% bias ratio unbiased (test)
98.13Accuracy
11
CMNIST 2% bias ratio unbiased (test)
97.15Accuracy
11
CMNIST 1% bias ratio unbiased (test)
95.35Accuracy
11
CMNIST correlation shift (test)
49.73Accuracy
10
CMNIST original unshifted
73.6Worst Accuracy
9
CMNIST v1 (test)
100Accuracy (Shift 1.0)
9
CMNIST (test)
94.3WGA
8
CMNIST (test)
69Accuracy
8
CMNIST In-distribution
99.5Accuracy
7
CMNIST (test)
0.948DCI
7