HotpotQA
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
HotpotQA
43.5EM
7
HotpotQA N=60 (test)
1Average Rounds
7
HotpotQA l1
0.05Cost per Pass
7
HotpotQA (test)
79.85Accuracy
7
HotpotQA
67.8EM
7
HotpotQA
63Accuracy
7
HotpotQA 200 held-out questions
91.5Accuracy
7
HotpotQA distractor Noise rho=0.5
64.5Accuracy (Acc)
7
HotpotQA distractor (Redundancy rho=0.5)
72.5Accuracy
7
HotpotQA distractor (Clean)
69Accuracy
7
HotpotQA
81AUROC
7
HotpotQA
52Occ
7
HotpotQA tool-augmented 1 (test)
72.8EM
7
HotpotQA
35.6EM
7
HotpotQA Shuffle-Hotpot-Full
31.5F1
7
HotpotQA Shuffle-Hotpot-Dist
41.9F1 Score
7
HotpotQA Full
32F1 Score
7
HotpotQA Dist
44.1F1 Score
7
HotpotQA
89.2Accuracy
7
HotpotQA fixed sample of 200 (dev)
26.62F1 Score
7
HotpotQA (400 randomly sampled instances)
66Exact Match (EM)
7
HotPotQA
60.3Accuracy
7
HotpotQA
65.31Precision
7
HotpotQA Extreme
23.59F1 Score
7
HotpotQA Noisy
28.76F1 Score
7