ReClor
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
ReClor (test)
80.6Accuracy
87
ReClor
80Accuracy
54
ReClor (dev)
0.786Accuracy
46
ReClor
85.43Accuracy
39
ReClor Hard (test)
87.2Accuracy
37
ReClor Easy (test)
86.4Accuracy
28
ReClor (test-H)
75Accuracy
23
ReClor (test-e)
91.8Accuracy
23
ReClor v1 (test)
78.1Accuracy
23
ReClor (test)
4.4ECE
21
ReClor (val)
84.4Accuracy (ReClor val)
15
ReClor 1.0 (test-H)
72.9Accuracy
13
ReClor OOD
87.8Accuracy
10
ReClor (ID)
90.1Accuracy (ReClor ID)
10
ReClor
61.8Pass@1
10
ReClor EASY v1
81.4Accuracy
9
ReClor 1.0 (test-E)
79.09Accuracy
8
ReClor 1.0 (val)
68.4Accuracy
8
ReClor v1 (test-e)
85.2Accuracy
5
ReClor v1 (dev)
81.8Accuracy
5
ReClor
36.67Accuracy
2
ReClor 300 examples (val)
2.7Accuracy
1