ZebraLogic
Benchmarks
TaskDatasetSOTA resultTrendResults
ZebraLogic v1.0 (test)
97.7Cell Accuracy
90
ZebraLogic (test)
92.2Grid Accuracy
90
ZebraLogic
98.8Accuracy
86
ZebraLogic
96Accuracy
49
Zebralogic
96.1Score
42
ZebraLogic
90.97Score
31
ZebraLogic
96.3Accuracy (%)
29
ZebraLogic
60.8Accuracy
25
ZebraLogic
348.71AUPC
20
ZebraLogic (held-out)
71.7Accuracy
18
ZebraLogic
44.5Avg Accuracy@1
12
ZebraLogic
0.817Avg Accuracy @1
11
ZebraLogic
96.8Easy Score
9
ZebraLogic
100Schema Validity
8
ZebraLogic
93.5Accuracy
6
ZebraLogic
96.3Accuracy (Avg@8)
6
Zebralogic
77Pass@8
6
ZebraLogic
4.01Error Rate (%)
6
ZebraLogic 140 puzzles
130Puzzle Accuracy
5
ZebraLogic
96.1Mean@1
4
ZebraLogic
2,245.5TPS
3
Zebralogic
78.7Accuracy (Zebralogic)
3
ZebraLogic mc_mode, Rows/Cols <= 4
73.73Average Score @64
1