LogiQA
Benchmarks
TaskDatasetSOTA resultTrendResults
LogiQA
78.9LogiQA Accuracy
251
LogiQA (test)
86Accuracy
153
LogiQA-2
83.8Accuracy
116
LogiQA
80.4Accuracy
106
LogiQA
50.23Accuracy
98
LogiQA (val)
58.37Accuracy
50
LogiQA (dev)
47.3Accuracy
40
LogiQA
60.22Accuracy
34
LogiQA
76.75Task Success Rate (TSR)
30
LogiQA2
86.4Accuracy
24
LogiQA original (test)
43.16Accuracy
22
LogiQA
0.039ECE
21
LogiQA
29.8Accuracy
21
LogiQA
50.94Accuracy
20
LogiQA
46.6Acc@t1
20
LogiQA (out-of-distribution)
8ECE
18
LogiQA
0.88Pass@1 Accuracy
18
LogiQA
67.75Accuracy
18
LogiQA
44.29Accuracy
17
LogiQA
48.61Pass@1 Accuracy
14
LogiQA
68.9Accuracy (LogiQA)
12
LogiQA (test)
85.75Accuracy
12
LogiQA n=8678 (test)
100Activation Score
11
LogiQA
74.1Accuracy
11
LogiQA 1.0 (test)
86Accuracy
11