GrailQA
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
GrailQA
94.4Hits@1
68
GrailQA Freebase (test)
86.9Hits@1
48
GrailQA
89.3Hits@1
40
GrailQA (dev)
89.5Overall F1
34
GrailQA v1.0 (test)
77.45Overall EM
33
GrailQA
78.82Accuracy
32
GrailQA (test)
91Overall Score
27
GrailQA (test)
91.76F1
27
GrailQA (Overall)
86.4Hits@1
20
GrailQA 500-sample (dev)
84.7F1 Score
18
GrailQA Zero-shot
89.1Hits@1
17
GrailQA Compositional
80Hits@1
17
GrailQA I.I.D.
92Hits@1
17
GrailQA Compositional (dev)
87F1 Score
14
GrailQA i.i.d. (dev)
93.8F1
14
GrailQA
80.4Hits@1
11
GrailQA
77.03Exact Match (EM)
9
GrailQA
89.3Hit@1
9
GrailQA v1.0 (dev)
83.4F1
9
GrailQAbility answerable zero-shot
78.01F1 (L)
8
GrailQAbility answerable (IID)
89F1(L)
8
GrailQA IID
92.4F1 Score
6
GrailQAbility unanswerable questions Zero-Shot (test)
88.31F1(R)
4
GrailQAbility unanswerable questions (test IID)
97.01F1(R)
4
GrailQA (val)
83.33Overall F1
4