MNIST
Benchmarks
TaskDatasetSOTA resultTrendResults
MNIST FedAvg 0.962 (test)
96.2Accuracy
88
MNIST
99.35AUC
87
MNIST VAE m=32 latent space (test)
98.29Accuracy
85
MNIST Decentralized, 2 m.n. (test)
96.11Accuracy
84
MNIST Decentralized 1 m.n. (test)
97.23Accuracy
84
MNIST Decentralized 0 m.n. (test)
98Accuracy
84
MNIST Centralized (test)
98.61Accuracy
84
MNIST
97.8Accuracy
82
MNIST 0.5 (test)
96.28Accuracy
80
Permuted Sequential MNIST (pMNIST) (test)
98.76Accuracy
79
MNIST -> USPS (test)
99.5Accuracy
79
MNIST
0.3Structural MSE
75
MNIST 5 tasks (test)
96.5Accuracy
75
MNIST
1.9Estimation Error
72
MNIST
99.64Clean Accuracy
71
MNIST
99.76Accuracy
70
permuted MNIST (pMNIST) (test)
97.87Accuracy
69
MNIST (test)
-3,408.16NLL (bits/dim)
69
MNIST standard (test)
99.7Accuracy
69
MNIST (test)
96.3Macro F1
68
MNIST
100Model Accuracy
66
MNIST (test)
99.6AUC
65
MNIST -> USPS (test)
98.5Accuracy
65
MNIST NN (test)
16Communication Rounds
62
Permuted MNIST T=784 (test)
97Accuracy
62