SQuAD
Benchmarks
TaskDatasetSOTA resultTrendResults
SQuAD v1.1 (dev)
95.8F1 Score
380
SQuAD v1.1 (test)
95.4F1 Score
260
SQuAD 2.0
89.4F1
215
SQuAD v2.0 (dev)
91.2F1
163
SQuAD
89.8F1
162
SQuAD (test)
91.2F1
156
SQuAD
0.89AUROC
127
Inj-SQuAD
0.11Combined ASR
123
SQuAD v1.1
94.7F1
85
SQuAD
93.33Exact Match
83
SQuAD (dev)
91F1
74
SQuAD
0.112ACE (General)
70
SQuAD v1.1 (val)
96.22F1 Score
70
SQuAD
94.7F1 Score
63
SQuAD
90.68F1 Score
58
SQuAD
89.9EM
58
SQuAD 2.0 (dev)
88.8EM
57
SQuAD 2.0 (test)
89.6EM
51
SQuAD (test)
83.8AUROCr
48
SQuAD 1.1 (dev)
89.71EM
48
SQuAD 1.1 (test)
89.898EM
46
SQuAD (test)
89GPT Judge Accuracy
45
SQuAD
85.5AUC
40
SQuAD
75.91Attack Accuracy
40
SQuAD
0.883AUC
39