ReClor
Benchmarks
TaskDatasetSOTA resultTrendResults
ReClor (test)
80.6Accuracy
87
ReClor
80Accuracy
54
ReClor (dev)
0.786Accuracy
46
ReClor
85.43Accuracy
39
ReClor Hard (test)
87.2Accuracy
37
ReClor Easy (test)
86.4Accuracy
28
ReClor (test-H)
75Accuracy
23
ReClor (test-e)
91.8Accuracy
23
ReClor v1 (test)
78.1Accuracy
23
ReClor (test)
4.4ECE
21
ReClor (val)
84.4Accuracy (ReClor val)
15
ReClor 1.0 (test-H)
72.9Accuracy
13
ReClor OOD
87.8Accuracy
10
ReClor (ID)
90.1Accuracy (ReClor ID)
10
ReClor
61.8Pass@1
10
ReClor EASY v1
81.4Accuracy
9
ReClor 1.0 (test-E)
79.09Accuracy
8
ReClor 1.0 (val)
68.4Accuracy
8
ReClor v1 (test-e)
85.2Accuracy
5
ReClor v1 (dev)
81.8Accuracy
5
ReClor
36.67Accuracy
2
ReClor 300 examples (val)
2.7Accuracy
1