MNIST
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MNIST
34Misclassification Rate
2
MNIST (train)
82.1Accuracy
2
ROT.MNIST (test)
0.03MSE
2
MNIST
98.99Accuracy (q=0.1)
2
MNIST (one vs. rest)
96Mean AUC
2
MNIST (non-IID (Dirichlet))
0.36Utility (Agent 0)
2
MNIST (train)
44.643Training Delta R
2
MNIST
0.998Pearson Correlation
2
MNIST Unbalanced (test)
96.86Accuracy
2
MNIST Balanced (test)
97.95Accuracy
2
Triggered MNIST
94.4Accuracy
2
MNIST conv computation graph
0.05Edge Cut
2
MNIST (mini-batch)
0.11Time (s)
2
MNIST Unsupervised
0.043Error Rate
2
MNIST all labels (test)
36Test Error (%)
2
MNIST 10% split (val)
102.5NLL
2
MNIST with low-rank interactions
90Accuracy
1
MNIST Downsampled to 2x2 (digits 6, 9)
98.3Accuracy
1
MNIST 784
0.013Average K
1
MNIST
-1Collusion Resistance
1
Rotated MNIST
0.802Spearman's ρ
1
MNIST Split
0.738Spearman Correlation (rs)
1
MNIST Cross-Device 100 clients
1Average TPR
1
MNIST Cross-Silo, 10 clients
100Average TPR
1
MNIST
61Robustness Ratio (roll)
1