2WikiMultiHopQA
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
2WikiMultihopQA OOD
39.2Exact Match Accuracy
15
2WikiMultihopQA
58.6F1 Score
15
2WikiMultiHopQA out-of-domain (val test)
51.7Exact Match (EM)
15
2WikiMultiHopQA
69.4Containment Accuracy
14
2WikiMultihopQA
44.4Exact Match (EM)
14
2WikiMultihopQA
56.91F1 Score
14
2WikiMultiHopQA
69.9String-F1
14
2WikiMultiHopQA in-domain (test)
69.8Accuracy (Response)
14
2WikiMultiHopQA December 2018 Wikipedia dump (test)
28.6EM
14
2WikiMultiHopQA
46.2Accuracy
13
2WikiMultiHopQA
0.58Average Inference Time (s)
13
2WikiMultihopQA
40.57Recall
13
2WikiMultiHopQA 1,000 queries (test)
71.1EM
13
2WikiMultiHopQA
3.28Speedup Factor
12
2WikiMultihopQA
82.1Accuracy
12
2WikiMultihopQA
82Pass@1
12
2WikiMultihopQA
3.57Prefilling Speedup Ratio
12
2WikiMultiHopQA 1,000 questions sample (val)
71.2Containment Accuracy
11
2WikiMultihopQA (val)
68.02F1 Score
11
2WikiMultiHopQA out-domain (test)
56.32LasJ
11
2WikiMultiHopQA
50.39F1 Score
10
2WikiMultihopQA
0.58LLM-as-a-Judge Score
10
2WikiMultihopQA
48.44Exact Match (EM)
10
2WikiMultiHopQA 15 (test)
67.25F1 Score
10
2WikiMultihopQA
0.06Latency (s/query)
9