DROP
Benchmarks
TaskDatasetSOTA resultTrendResults
DROP Single Client
59.01Accuracy (DROP Single Client)
6
DROP
82.86Score
6
DROP (dev)
85.2EM
6
DROP CS
54.2Score
6
DROP
37.9MRR
6
DROP
82.4Pass@1
5
DROP history
68.52F1 Score
5
DROP (test)
83.74EM
4
DROP Football
69.98Accuracy
4
DROP Non-football
79.24Accuracy
4
DROP
82.55Accuracy
4
DROP numerical reasoning Football 500 randomly sampled cases (test)
63.4Accuracy
4
DROP numerical reasoning Non-football 500 randomly sampled cases (test)
74.2Accuracy
4
DROP numerical problems (val)
82.45Non-football Accuracy
4
DROP span-subset (dev)
79.8EM
4
DROP (test)
35.13Accuracy
3
DROP
18.95Throughput (TPS)
3
DROP
80.4Accuracy
3
DROP
68.7AUROC
2
DROP
2.8Grammar
2
DROP Football (test)
—Accuracy
0
DROP Non-football (test)
—Accuracy
0