SQuAD
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
SQuAD v1.1 (dev)
38.6EM
13
SQuAD In-Domain (IND-MGT)
93.15Accuracy
12
SQuAD-32K
80Score
12
SQUAD
93.8Accuracy
12
SQuAD
75.5Accuracy (Acc)
12
SQuAD
5,303Inference Throughput (Samples/sec)
12
SQuAD v1.1 (test)
89.24EM
12
SQuAD
73.3Hit@20
12
SQUAD v2
28.9Correct Count
12
SQuAD
86.57F1 Score
12
SQuAD Dir alpha=0.1 v2
19.91F1 Score
12
SQuAD Dir alpha=0.1 v2 Standard
21.25F1 Score
12
SQuAD (dev)
91.2F1 Score (F1)
12
adversarial SQuAD (test)
46.6Add Sent Score
12
SQuAD 2.0
86.5Performance (F1/EM)
11
SQuAD 2.0
86.5Performance
11
SQuAD
58F1 Score
10
SQuAD
86.35F1 Score
10
SQuAD 2.0
1.66Factual Accuracy
10
SQuAD 2.0
5.1BLEU-1
10
SQuAD v2.0
50.64Disambiguation Score
10
SQuAD v1.1 (test)
80.02F1 Score
10
SQuAD v2
19.53EM
10
SQuAD v2
95.1TGS
10
SQuAD 2.0
90.8Score
10