PopQA
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
PopQA
19.85ECE
10
PopQA single-hop (test)
47.2Avg@16
10
PopQA (test)
50.2EM
10
PopQA (All)
32.9Accuracy
10
PopQA
73.84Accuracy
9
PopQA
46.6Recall@2
9
PopQA
23EM
9
PopQA
0.908AUROC
9
PopQA
46.86EM
8
PopQA
94.27Correct First (CF)
8
PopQA
50.8F1 Score
8
PopQA (test)
56.2R@5D
8
PopQA
33.3F1 Score
8
popQA Normal (test)
25.95F1 Score
8
PopQA Wikipedia-18 retriever context (test)
40.34EM
7
PopQA
43.31F1 Score
7
popQA Extreme
24.96F1 Score
7
popQA Noisy
25.07F1 Score
7
POPQA Frequent
89.9R@10
7
POPQA Infrequent
76.8Recall@10
7
POPQA Frequent
94.5Recall@10
7
POPQA Infrequent
80.8Recall@10
7
PopQA
39.7EM
7
PopQA
30,894.5Avg Token Count
7
PopQA
46.7LLM Judge Score
6