PopQA
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
PopQA Non-Factoid
51.4GSM8K Accuracy
6
PopQA Factoid
86.6ID Score
6
PopQA
38.36Accuracy
6
PopQA sampled queries from Augmented dataset
0.873Average Latency (s)
6
PopQA (Evaluation)
11.2Accuracy
6
PopQA
85.8LAMA Score
5
PopQA
50.8Accuracy
5
PopQA
23EM
5
PopQA
49.8subEM
5
PopQA 150 questions
33Base SR
5
PopQA
70Match
5
PopQA (test)
36.34Exact Match
5
PopQA
26.4Joint Answer-and-Evidence Score
4
PopQA
39.2Evidence Score
4
PopQA
50.4Mean Performance
4
PopQA
49.1CoverEM
4
PopQA
3.337Recall@1
4
PopQA
44.5EM
4
PopQA
21.2Exact Match
4
PopQA (test val)
52.1Exact Match (EM)
4
PopQA (test)
71.43Cost Reduction
3
PopQA OOD transfer
62.9Overall EM
3
PopQA
10.1Brier Score (BS)
3
PopQA
30.59Accuracy
3
PopQA
71.26Pattern-based Score
3