ColorMNIST
Benchmarks
TaskDatasetSOTA resultTrendResults
ColorMNIST
100Adversarial Accuracy
30
ColorMNIST
100Adversarial Accuracy
30
ColorMNIST (ρ = 80%) (test)
76.6Average Accuracy
20
ColorMNIST (OOD)
73.8Accuracy
16
ColorMNIST (ID)
100Accuracy
16
ColorMNIST reversed (test)
50.93Accuracy
13
ColorMNIST
206MSE
10
ColorMNIST (ρ = 80%)
76.6Average Accuracy
7
ColorMNIST (test)
89.9Accuracy
6
ColorMNIST 40 OOD compositions (Generalization)
84.7FID
5
ColorMNIST Faithfulness 40 OOD compositions
62.6FID
5
ColorMNIST+
99.7Task Accuracy
5
ColorMNIST+
99.2Concept Accuracy
4