NATS-Bench
Benchmarks
TaskDatasetSOTA resultTrendResults
NATS-Bench SSS ImageNet16-120 (stratified subset)
85.6Mean Abs Kendall Corr Coeff
24
NATS-Bench SSS CIFAR-100 (stratified subset)
3.4mAKCC
24
NATS-Bench SSS CIFAR-10 (stratified subset)
88.8mAKCC
24
NATS-Bench SSS ImageNet-16-120 (test)
46.2Accuracy
20
NATS-Bench SSS CIFAR-100 (val)
73.49Accuracy
20
NATS-Bench SSS ImageNet-16-120 (val)
46.73Top-1 Accuracy
13
NATS-Bench SSS CIFAR-100 (test)
73.51Top-1 Accuracy
13
NATS-Bench SSS CIFAR-10 (test)
94.37Top-1 Acc
13
NATS-Bench SSS CIFAR-10 (val)
91.61Top-1 Accuracy
13
NATS-Bench TSS ImageNet16-120 Stratified Subset
84.7Mean Abs Spearman Correlation
12
NATS-Bench TSS CIFAR-100 stratified subset
0.885Spearman Correlation (Abs)
12
NATS-Bench TSS (Topology Search Space) CIFAR-10 stratified
92.8Mean Abs Spearman Correlation
12
NATS-Bench SSS (Size Search Space) ImageNet16-120 stratified subset
0.971Mean Absolute Spearman Correlation Coefficient
12
NATS-Bench SSS (Size Search Space) CIFAR-100 stratified subset
0.924mAbs Spearman Correlation
12
NATS-Bench SSS (Size Search Space) CIFAR-10 stratified subset
98.2Spearman Correlation (x100)
12
NATS-Bench TSS ImageNet16-120 (non-stratified)
54.7Mean Absolute Kendall Correlation
12
NATS-Bench TSS CIFAR-100 (non-stratified)
0.609mAbs Kendall Correlation
12
NATS-Bench TSS CIFAR-10 (non-stratified)
0.611Mean Absolute Kendall Correlation
12
NATS-Bench SSS ImageNet16-120 (non-stratified)
0.795mKendall Corr (abs)
12
NATS-Bench SSS CIFAR-100 (non-stratified)
4.4mAbs Kendall Corr
12
NATS-Bench SSS CIFAR-10 (non-stratified)
0.768mAbs Kendall Corr (x100)
12
NATS-Bench ImageNet16-120 (test)
46.58Accuracy
10
NATS-Bench CIFAR-100 (test)
73.01Accuracy
10
NATS-Bench CIFAR-10 (test)
94.34Accuracy
10
NATS-Bench ImageNet16-120 TSS (test)
46.34Accuracy
8