MNIST
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MNIST
0.57Error Rate
18
MNIST (test)
0.003Loss
18
MNIST Dirichlet shift alpha=10 (val size=2000)
440MSE
17
MNIST Dirichlet shift alpha=10 size=1500 (val)
480MSE
17
MNIST Dirichlet shift alpha=10 (val size=500)
0.6MSE (10^-3)
17
MNIST Dirichlet shift alpha=0.1 (val)
230MSE
17
MNIST Dirichlet shift alpha=0.1 size=1500 (val)
280MSE (x 10^-3)
17
MNIST Dirichlet shift alpha=0.1 size=500 (val)
970Mean Squared Error (MSE)
17
mnist
1.36CPU Time
17
MNIST (test)
1.46FID
17
MNIST (labelled)
99.2Accuracy
17
MNIST (test)
3.53Average Time (s)
17
MNIST first 50 samples (test)
16Failing Samples
16
MNIST 1.0 (Fourth Session Transition)
94.51Proposed Score
16
MNIST 1.0 (Second Session Transition)
93.99Proposed Score
16
MNIST
298Mean Failure Count
16
MNIST Heldout Digit 9
85.4AUPRC
16
MNIST Heldout Digit 7
0.857AUPRC
16
MNIST Heldout Digit 5
95.9AUPRC
16
MNIST Heldout Digit 4
93.5AUPRC
16
MNIST (Heldout Digit 1)
95.9AUPRC
16
MNIST
3.91Performance Degradation (PD)
16
MNIST
100Precision
16
MNIST (test)
98.2Accuracy
16
MNIST
2.5FID
16