NaturalQuestions
Benchmarks
TaskDatasetSOTA resultTrendResults
NaturalQuestions filtered (dev)
44.3EM
48
NaturalQuestions
60.65F1 Score
44
NaturalQuestions
62.56F1
42
NaturalQuestions (NQ)
52.88SubEM
40
NaturalQuestions 20 documents
2,946Tokens Processed
26
NaturalQuestions (test)
43.62EM
24
NaturalQuestions (NQ) (test)
82.24Top-20 Acc
23
NaturalQuestions processed
83.05Accuracy
22
NaturalQuestions Open
7.7Exact Match
12
NaturalQuestions standard (test)
45.5Accuracy
12
NaturalQuestions
0.005Latency (s)
11
NaturalQuestions (test)
59.98F1 Score
11
NaturalQuestions (NQ) v1.0 (test)
83Acc@20
11
NaturalQuestions (NQ) 5-shot
42.85EM Accuracy
10
NaturalQuestions (NQ) top 1000 samples (test)
40.3Exact Match
10
NaturalQuestions (test)
54.4Top-1 EM
9
NaturalQuestions (NQ) (test)
80.8R@2
8
NaturalQuestions (val)
5.7Exact Match
5
NaturalQuestions (250 questions)
61.4Suppress Rate
4
NaturalQuestions
0.433Accuracy
3