SQuAD
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
SQUAD 1.1 (Generalization)
28F1 Score
18
SQuAD
83.9ROC
18
SQuAD hidden 1.1 (test)
82.3EM
18
SQuAD 2.0 Sep 9, 2018 (test)
86.9EM
17
SQuAD In-domain (test)
92.2Accuracy
16
SQuAD
28.32EM
16
SQuAD MRQA 2019
92.2F1 Score
16
S-SQuAD (test)
64.1EM
16
SQuAD
50.2EM
16
SQuAD 1.1 (dev)
25.4BLEU-4
16
SQuAD v2
87.51Accuracy
15
SQuAD Extract
0.128Mean Per-Step Regret
15
SQuAD Abstract
0.155Mean Per-Step Regret
15
SQuAD
91.27Accuracy
15
SQuAD (dev)
0.922F1 Score
15
SQuAD Short-form QA
74.79F1 Score
14
SQuAD
85.34Accuracy
14
SQuAD
76.4Accuracy
14
SQuAD
83.06ROUGE-Lsum
14
SQuAD 20% coverage
99.04F1
14
SQuAD
0.983Grammatical Correctness
14
SQuAD
82.1MRR
14
SQuAD
0.721AUCs
14
SQuADShifts
0.9AUC
13
SQuAD
41.3Accuracy
13