PopQA
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
PopQA OOD
45.7Exact Match Accuracy
15
PopQA Gemini outputs (test)
78.7AUROC
15
PopQA GPT outputs (test)
0.802AUROC
15
PopQA Llama outputs (test)
0.77AUROC
15
PopQA out-of-domain (val test)
50.1Exact Match (EM)
15
PopQA
45.5Accuracy (EM)
14
POPQA
65.04fEM
14
PopQA 2018 Wikipedia dump (dev)
47.2Accuracy
14
POPQA Long-tail
74.5Recall@10
14
PopQA
62.545Accuracy
13
PopQA Pop.
27.7OSR Score
13
PopQA
46.3Exact Match (EM)
13
PopQA (test)
0.859AUROC
12
PopQA held-out 50% (test)
85.7AUROC
12
PopQA
47.4Accuracy
12
PopQA
66.3CEM (%)
12
PopQA
50.6Accuracy
11
PopQA (test)
53.8Recall@5
11
PopQA (test)
51.26EM Accuracy
11
PopQA
65.7Accuracy
11
PopQA (test)
0.435r_connect
10
PopQA D_special (test)
48.25r_pub Score
10
PopQA
47.87EM
10
PopQA
28.7EM
10
PopQA 1,000 queries (test)
50.94EM
10