HotpotQA
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
HotpotQA
62Answer Hit Rate (aHR)
12
HOTPOTQA Unfiltered
66.08Accuracy
12
HotpotQA
204TTFT (ms)
12
HotpotQA
0.66Peak GPU Memory (GB)
12
HotpotQA distractor setting
67.8F1 Score
12
HotpotQA
0.673Exact Match (EM)
12
HotpotQA
76.18Answer F1
12
HotpotQA
61.63F1 Score
12
HotpotQA random subset of 10,000 samples
0.024Log-Probability Drop
12
HotpotQA
4.1Log-Prob Drop
12
HotpotQA
89.33SASR
12
HotPotQA (test)
25.52Accuracy
12
HotpotQA
60.8Pass@1
12
HotpotQA
64.2CEM
12
HotpotQA
17,910Output Tokens
12
HotpotQA (test)
72.79AUPRC
12
HotpotQA v1.0 (test)
79.16Exact Match (EM)
12
HotpotQA
53Accuracy
12
HotpotQA
68.2NDCG@10
12
HotpotQA D (eval)
46.2Accuracy
12
HotpotQA D (train)
95.4Accuracy
12
HotpotQA (train)
100Accuracy
12
HotpotQA
496Time to Last Token (ms)
12
HotpotQA
60.94EM (LQ)
12
HotpotQA in-corpus setting (test)
23Accuracy
12