PopQA
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
PopQA n=1000 (test)
0.895AUROC
24
PopQA out-of-domain
52.4Accuracy
24
PopQA longtail
45.96EM
23
PopQA
51.75Strict EM Accuracy @5
21
PopQA
49.8Success Rate
21
PopQA D_special (test)
30.69r_pri
20
PopQA
47.9EM
20
PopQA
67.08AUPRC
20
PopQA
52.3FAR (Overall)
19
PopQA
65.5R@5
19
PopQA 128K
23Accuracy
18
PopQA out-of-domain
46.9Exact Match
18
PopQA
58.4pass@1
18
PopQA
48.8Accuracy
18
PopQA
50.5F1 Score
17
PopQA
9.6Brier Score
17
PopQA
41.6EM
17
PopQA
72.38Accuracy
16
PopQA fixed split (test)
50.78Exact Match (EM)
16
PopQA
0.018ECE
16
PopQA (Frequent)
52.7Exact Match (EM)
16
PopQA Infrequent
42.9Exact Match Accuracy
16
PopQA
90.14P@1
16
PopQA
83.09A1
16
PopQA v1.0 (test)
83.07A1
16