HotpotQA
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
HotpotQA
76Exact Match (EM)
28
HotpotQA (test)
38.8Unreliability Rate
28
HotpotQA (test)
68.6Accuracy
28
HotpotQA
5ASR
28
HotpotQA (test)
50.18Accuracy (Exact Match)
27
HotpotQA (ID)
0Degeneration Count
27
HotpotQA (ID)
2.41Exact Match (EM)
27
HotpotQA
48.1EM (%)
27
HotpotQA IRCoT (500 samples) (test)
54.6ACC
27
HotpotQA (test)
59F1 Score
26
HotpotQA
67.19AUROC
26
HotpotQA v1.1 (test)
56.94Easy Score
26
HotpotQA
86.7LLM Accuracy
25
HotpotQA
80.3Score
25
HotpotQA
56.2F1 Score
25
HotpotQA
76.8Abstain Accuracy
25
HotPotQA
66ACC1
25
HotpotQA
59.64Accuracy
24
HotpotQA
36.4EM
24
HotpotQA
70.4F1 Score
24
HotpotQA online Google Search API (test val)
51.3Exact Match (EM)
24
HotpotQA offline Wiki-18 (test val)
44.7EM
24
HotpotQA (test)
37.623EM
24
HotpotQA (val)
92ASR
24
HotpotQA
96.6ASR (Substring-based)
24