Natural Questions
Benchmarks
TaskDatasetSOTA resultTrendResults
NQ (Natural Questions) in-domain (test)
31.45EM
20
Natural Questions (test)
98.4Pc
20
Natural Questions
11.4Macro EM
19
Natural Questions
45.72EM
18
Natural Questions (test)
48.78EM
18
Natural Questions BEIR
73.9NDCG@10
17
Natural Questions (NQ) fixed split (test)
38.34Exact Match (EM)
16
Natural Questions
80.34D0 Score
16
Natural Questions (NQ) 5-shot
38.8Accuracy
16
Natural Questions (NQ) benign retrieval
94SR (%)
16
Natural Questions (test)
59.83Recall@1
16
Natural Questions (NQ) (test)
67.4F1 Score
16
Natural Questions
5.803Speedup Ratio
15
Natural Questions
47.8Accuracy
15
NQ (Natural Questions)
25.2OSR (%)
13
Natural Questions NQ320k
0.685HR@1
13
NQ (Natural Questions)
86.1Hit@20
12
Natural Questions
40.1Accuracy
12
Natural Questions (NQ)
83.02Accuracy
11
Natural Questions (NQ) long answer (dev)
76.5F1 Score
11
Natural Questions
2.6Average Rank
10
Natural Questions
80LLM-as-a-Judge Score
10
Natural Questions (NQ)
28.3Exact Match (EM) Accuracy
10
Natural Questions QA
162.54Decoding Speed (tokens/s)
10
Natural Questions (NQ)
37.8Exact Match
10