ZebraLogic
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
ZebraLogic v1.0 (test)
97.7Cell Accuracy
90
ZebraLogic (test)
92.2Grid Accuracy
90
ZebraLogic
98.8Accuracy
86
ZebraLogic
96Accuracy
49
Zebralogic
96.1Score
42
ZebraLogic
90.97Score
31
ZebraLogic
96.3Accuracy (%)
29
ZebraLogic
60.8Accuracy
25
ZebraLogic
348.71AUPC
20
ZebraLogic (held-out)
71.7Accuracy
18
ZebraLogic
44.5Avg Accuracy@1
12
ZebraLogic
0.817Avg Accuracy @1
11
ZebraLogic
96.8Easy Score
9
ZebraLogic
100Schema Validity
8
ZebraLogic
93.5Accuracy
6
ZebraLogic
96.3Accuracy (Avg@8)
6
Zebralogic
77Pass@8
6
ZebraLogic
4.01Error Rate (%)
6
ZebraLogic 140 puzzles
130Puzzle Accuracy
5
ZebraLogic
96.1Mean@1
4
ZebraLogic
2,245.5TPS
3
Zebralogic
78.7Accuracy (Zebralogic)
3
ZebraLogic mc_mode, Rows/Cols <= 4
73.73Average Score @64
1