SciQ
Benchmarks
TaskDatasetSOTA resultTrendResults
SciQ
97.2Accuracy
283
SciQ
97.7Normalized Accuracy
137
SciQ
94.3Accuracy (SciQ)
134
SciQ
100Accuracy
91
SciQ
64.8AUROC
80
SciQ
60PRR
66
SciQ
86.1ROC-AUC
66
SciQ
87.79AUC
51
SciQ (test)
35.01AURC
42
SciQ
77.99AUROC
42
SciQ
88.99AUC
42
SciQ (test)
92.9Accuracy
36
SciQ (train)
100Accuracy
36
SciQ
0.933AUROC
33
Sciq
86.4Acc Norm
32
SciQ
93.7Accuracy
32
SciQ (test)
74.5AUROC
28
SciQ 1,000 samples (val)
62.6AUROC
27
SciQ
97.08Accuracy
25
SciQ In-Domain (test)
83.68Precision
24
SciQ
98.3First-Token Accuracy
24
SciQ (ID)
76.44Precision
24
SciQ
90.9Accuracy
22
SciQ
4.42ECE@1
21
SciQ
5.21ECE
21