Synth
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
Synth-50
100Final Mean Accuracy
26
Synth-10
100Final Mean Accuracy
26
Synth4K-5
88.5Delta Threshold Accuracy (0.5)
16
Synth4K-3
82Acc (delta=0.5)
16
Synth-Large synthetic (test)
2,160.47Objective Value
14
Synth-Medium (test)
1,387.82Objective Value
14
Synth Small (test)
811.01Objective Value
14
Synth (basis+std, r = 1) OOD
61Accuracy
14
Synth basis+std r = 1 ID
62.5Accuracy
14
SYNTH
1MCC
14
Synth-12 sigma=0.9
0.36MSE
12
Synth sigma=0.7 12-variable synthetic
0.379MSE
12
Synth-12 sigma=0.5 synthetic
0.391MSE
12
Synth synthetic sigma=0.3 12-variable
0.378MSE
12
Synth 12-variable synthetic (sigma=0.1)
0.386MSE
12
SYNTH
86.9Accuracy
12
Synth-50
0Average Forgetting
11
Synth factual conditions (test)
0.54MAE
9
SYNTH
100Accuracy
8
Synth
0.02Latency (s)
7
Synth
468.29Memory Consumption (MB)
7
Synth-B
2.33PMSE
6
Synth
100IR
5
SYNTH
7,371,212Yield
4
Synth (test)
9.6tcpWER
3