MNIST
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MNIST binary subsample (test)
4.2AUC-DEL Score
11
MNIST
0.065Empirical Risk (Rs(h))
11
MNIST
99.22Benign Accuracy
11
RotatedMNIST
96.7Accuracy (0°)
11
MNIST pathological non-IID (500 clients, 96 samples)
0.994Top-1 Acc
11
MNIST pathological non-IID (100 clients, 480 samples)
99.38Top-1 Acc
11
MNIST pathological non-IID (50 clients, 960 samples)
99.48Top-1 Acc
11
MNIST symmetric noise 0.6 Dirichlet non-IID (a=100) (test)
30ECE
11
MNIST symmetric noise 0.2 Dirichlet non-IID (a=100) (test)
27ECE
11
MNIST pair noise 0.4 Dirichlet non-IID (a=100) (test)
0.28ECE
11
MNIST pair noise 0.1 Dirichlet non-IID (a=100) (test)
14ECE
11
MNIST original unenhanced (test)
98.6Accuracy (%)
11
Irregular sequential MNIST (test)
95.73Accuracy
11
MNIST bg-img-rot (test)
0.355Error Rate (%)
11
MNIST binarized (test)
78.43NLL
10
MNIST (test)
98.1Accuracy
10
MNIST transformed (test)
99.2Accuracy
10
MNIST
86.95Average Accuracy
10
MNIST (last 5% of data stream)
95.02Accuracy
10
MNIST last 20% of data stream
91.77Average Accuracy
10
MNIST (Entire data stream)
86.95Accuracy
10
Permuted MNIST 10 tasks
98.6Continual Accuracy
10
MNIST
96Neighbor Recall
10
Rotated MNIST
85.64AutoAttack
10
Permuted MNIST (test)
81.6AutoAttack Robust Accuracy (%)
10