SQuAD
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
SQuAD (out-of-domain)
99.8Accuracy
3
SQUAD
94Accuracy
3
SQuAD
92ROC AUC
3
SQuAD GPT-Neo-2.7B
53.33TPR @ FPR=1%
3
SQuAD less-mix (test)
0.091F1 Score
3
SQuAD 2.0
9.7Exclusive Catch Rate (@10%)
3
SQuAD v2
89.3Accuracy
3
SQuAD Extract
75.9Accuracy
3
SQuAD Abstract
63.6Accuracy
3
SQuAD v2 (test)
60.8EM
3
SQuAD
91.9ROUGE-L Recall
3
SQuAD
91Accuracy
3
SQuAD
114.73Latency (s)
3
SQuAD v2 (250 QA pairs)
4.54Relaxed F1
3
SQuAD
74.3Context Relevance
3
SQuAD
57.63EM
3
SQuAD 2.0
12.94Throughput (TPS)
3
SQuAD
49.7Diversity Win
3
SQuAD All Wikipedia (test)
29.8Top-1 EM Accuracy
3
SQuAD
14.8PPE
2
SQuAD 2-doc (n=293)
68.9R@1
2
SQuAD 1-doc
91.7R@1
2
SQuAD
59.4F1 Score
2
SQuAD 2.0
69.04Coverage@0.9
2
SQuAD (train)
40.02Exact Match
2