HotpotQA
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
HotpotQA (test)
98.2F1
5
HotpotQA (test)
48.71Accuracy
5
HotpotQA
124,460Triplets Count
5
HotpotQA
0.33Avg Running Time (s)
5
HotpotQA (test)
60Exact Match (EM)
5
HotpotQA (test)
0.77LLM Judge Score
5
HotpotQA
29.1F1 Score
5
HotpotQA Dynamic Numeric Reasoning suite (randomly sampled 200 multi-hop numeric reasoning questions)
2,252Tokens (No-adv)
5
HotpotQA
60.6Accuracy
5
HotpotQA (test)
66.7EM
5
HotpotQA (test)
68.75EM
5
HotpotQA
52EM
5
HotpotQA L-M pair
40.58F1 Score
5
HotpotQA L-Q pair
43.16F1 Score
5
HotpotQA distractor (val)
21EM
5
HotpotQA (test)
65.7Accuracy
5
HotpotQA distractor setting (dev)
70.57All Score
5
HotPotQA MA
5ASR (Attack Success Rate)
4
HotPotQA-Vanilla In-Domain
0.54AURC
4
HotpotQA
6.17Average Response Latency (s)
4
HotpotQA single-predictor (test)
60.4Exact Match
4
HotpotQA single-predictor (val)
64Exact Match
4
HotpotQA
78.6Recall@2
4
HotpotQA
86.6Sel-AUC
4
HotpotQA (val)
80F1 Score
4