SAT
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
SAT n=50 planted (alpha=4.0)
100Solve Percentage
8
SAT
64.8Accuracy
8
SAT iid (val)
92.7Accuracy
8
SAT
0.893Accuracy
7
SAT (test)
75.33Accuracy
7
SAT (val)
93.48Accuracy
7
SAT
75.33Accuracy
7
SAT Synthetic
65.2Accuracy
6
SAT small
3.29Normalized Count
6
SAT Hard (test)
64Accuracy
6
SAT Medium (test)
95Accuracy (SAT Medium Test)
6
SAT Easy (test)
98Accuracy
6
SAT
85.1SAT Reading Score
6
SAT 66
63.6Accuracy
5
SAT n=75 random (alpha=4.26)
100Solve Rate
5
SAT n=50 random (alpha=4.26)
100Solve Rate
5
SAT4
4.84Error Rate
5
SAT (test)
71.9Accuracy
5
SAT
81.3Score
4
SAT-Real
88.7Average Accuracy
4
SAT (real)
77.33Accuracy
4
SAT (synthetic)
69.33Accuracy
4
SAT 1.0 (test)
65.3Accuracy (MCQ)
4
SAT
84.4Accuracy
4
SAT 8-20 variables (test)
100Success Rate (8 Variables)
3