LogiQA
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
LogiQA
1Normalized Generalization Error Approximation
10
LogiQA
1.1Generalization Error Approx. (G(W))
10
LogiQA Chinese
52.4Pass@1 Accuracy
10
LogiQA English
53Pass@1 Accuracy
10
LogiQA
48.75Task Success Rate (TSR)
9
LogiQA
60.22Accuracy
8
LogiQA 2.0 (test)
0.614Accuracy
8
LogiQA
78.5Accuracy
7
LOGIQA
56.4Hit@1 (LOGIQA)
7
LogiQA
22.12Accuracy
7
LogiQA CN
34.92Accuracy (LogiQA CN)
6
LogiQA
74.5Task Success Rate (TSR)
6
LogiQA
43.57Selection Accuracy
6
LogiQA v2 (test)
55.5Macro F1 Score
6
LogiQA 1.0 (val)
42.24Accuracy
6
LogiQA MCR 2.0 (test)
54.4Accuracy
4
LogiQA NLI 2.0 (test)
56.2Accuracy
4
LogiQA
5.771Causal Fidelity Score (CFS)
4
LogiQA v1 (dev)
49.6Accuracy
4
LogiQA
30.11Normalized Accuracy
2