SQuAD
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
SQuAD
0.83Wall-clock Time (hours)
8
SQuAD
80.22F1 Score (SQuAD v1.1)
8
SQuAD v1.1 v2.0 (test dev)
89.4F1 Score
8
SQuAD
78.5Accuracy (CA)
8
SQuAD
84.8ROUGE-L
8
SQuAD v0.2 (test)
0.464METEOR
8
SQuAD Num ID v1 (test)
73.7EM
8
SQuAD Rare ID v1 (test)
73.7EM
8
SQuAD Title ID v1 (test)
73.7EM
8
SQuAD
62.7AUPC
8
SQuAD
33,350.02Average Context Length (tokens)
8
SQuAD short
64.2Exact Match (EM)
7
SQuAD2 50% coverage
34.43F1 Score
7
SQuAD adv (50% coverage)
42.79F1
7
SQuAD (50% coverage)
97.01F1 Score
7
SQuAD-adv 20% coverage
43.99F1 Score
7
SQuAD (Zhao split)
26.3BLEU-4
7
SQuAD (test)
40.7Accuracy
7
SQuAD AddOneSent
60.9EM
7
SQuAD AddSent
53EM
7
SQuAD v1.1 (dev)
99.3Accuracy
7
SQuAD golden
96.6Accuracy
7
SQuAD (eval)
59.84EM (Clean)
6
SQuAD QA full profile (N = 200; incl. 50 calib.)
0FPR
6
SQuAD 1.1 (test)
72.73Rouge-L
6