OBQA
Benchmarks
TaskDatasetSOTA resultTrendResults
OBQA
94.95Accuracy
347
OBQA
89.2Accuracy
187
OBQA (test)
92.7Accuracy
90
OBQA
93.2Accuracy
79
OBQA
97.67Accuracy
46
OBQA
48Normalized Accuracy
44
OBQA (test)
44Normalized Accuracy
42
OBQA to MMLU
87.09AUROC
41
OBQA
35.2Zero-shot Accuracy
36
OBQA
44.2Accuracy
32
OBQA
41.8Accuracy (Normalized)
29
OBQA
88.03AUROC
24
OBQA
91.4First-Token Accuracy
24
OBQA (test)
71.6Accuracy
21
OBQA
93.4Accuracy
19
OBQA
31.4Accuracy
17
OBQA (dev)
86.1Accuracy
17
OBQA
86Accuracy
16
OBQA
83.08Accuracy
16
OBQA
88.8Accuracy
14
OBQA
95.2Accuracy
12
OBQA In-Distribution
88.43Accuracy
12
OBQA (leave-one-out setup)
87.7Average Accuracy
12
OBQA
90.1Accuracy
12
OBQA (out-of-domain)
95.59Acc
12