HotpotQA
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
HotpotQA out-of-domain
64.9Precision
12
HotpotQA
4.14Prefill Speedup Ratio
12
HotpotQA
0.686Average Score
12
HotpotQA
32Accuracy
12
HotpotQA
36.3Exact Match (EM)
12
HotpotQA distractor
90.08Support F1
12
HotpotQA Full Wiki hidden (test)
73F1
12
HotpotQA D^x (train)
92.6Accuracy
12
HotpotQA 1,000 questions sample (val)
66.3Containment Accuracy
11
HotpotQA
0.94Pearson r
11
HotpotQA
89.3Recall
11
HotpotQA
65.4Accuracy
11
HotpotQA (test val)
59.7F1 Score
11
HotpotQA
1.86Helmet Score
11
HotpotQA
42.6EM
11
HotpotQA
0Cost (cents)
11
HotpotQA (test)
0Cost (cents)
11
HotpotQA (test)
88Accuracy
11
HotpotQA
88.12Solve Rate
11
HotpotQA (val)
70.96F1 Score
11
HotpotQA in-domain (test)
61.59LASJ
11
HotPotQA (test)
45.6Answer Accuracy
11
HotpotQA Full Wiki hidden (test)
76.4F1 Score
11
HotPotQA full-wiki (test)
50.66Accuracy
10
HotpotQA (HpQA) Wiki18 index (test)
59.1Pass@4
10