SQuAD
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
SQuAD v1.1
1.34Latency Speedup
2
SQuAD fine-tuning 1.1
91.29F1 Score
2
SQuAD adaptation 2 (test)
1.8EM
2
SQuAD lost (test)
37.31F1 Score
2
SQuAD 1.1
77.93Top-1 Accuracy
2
SQuAD Human Evaluation Subset (test)
2.18Avg Score
2
SQuAD-derived (in-domain)
22.6F1 Score
1
SQuAD (test)
847Mean |C|
1
SQuAD (held-out samples)
89.7Coverage
1
SQuAD v1.1 (val)
—Primary metric
0
SQuAD MRQA 1.1
—F1
0