EMNIST
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
EMNIST (test)
0.917AUROC
14
EMNIST (test)
20.8FID (A)
13
EMNIST to MNIST (test)
4.23FID
13
EMNIST (Non-IID (4-Class partition))
86.88Accuracy
12
EMNIST ByClass (test)
2.3Bits Per Pixel (BPP)
12
EMNIST-Digits (test)
99.82Accuracy
12
EMNIST ByClass (test)
1.98Mean Prediction Set Size
12
EMNIST
2.19Avg Prediction Set Size
12
EMNIST 10 different calib. (test)
2.42Mean Prediction Set Size
12
EMNIST (test)
1.98Mean Prediction Set Size
12
EMNIST 20 tasks
1.9Backward Transfer
11
M(EMNIST-10, F-EMNIST)
94.37Accuracy
11
EMNIST transformed (test)
83.9Accuracy
10
EMNIST extended (test)
89.21Accuracy
10
EMNIST Cross-device
75.87Aθ Score
10
EMNIST Cross-silo
76.39Aθ
10
EMNIST alpha = 0.1
0.203Max r_k(theta)
10
EMNIST dir(alpha=0.1) (test)
0.063Max r_k(theta)
10
EMNIST Pathological heterogeneous (K = 20, IR = 10) (test)
97Accuracy
10
EMNIST Practical heterogeneous (alpha = 0.1, K = 20) (test)
96.84Accuracy
10
EMNIST (non-iid)
84.42Accuracy
10
EMNIST iid (test)
85.04Test Accuracy
10
EMNIST IID (test)
67.5Time Reduction (%) @ Acc 0.6
9
emnist
0.64AMI
9
EMNIST Non-IID
69.95Accuracy (Group 95-100%)
8