List your company
ResearchFormerly WizWand

Reading Comprehension on DROP (dev)

88.1F1 ScoreQDGATp
-2.088821.325644.7468.1544Mar 1, 2019Aug 17, 2019Feb 3, 2020Jul 22, 2020Jan 7, 2021Jun 26, 2021Dec 13, 2021
Updated 8mo ago

Evaluation results

MethodLinks
QDGATpEnsemble=true, Backbone=RoBERTa, Pre-training=SQuAD2020.0988.185.31
TASE_IO + SSE2019.0987.880.5
TASE_BIO + SSE2019.0987.179.7
QDGATpBackbone=RoBERTa, Pre-training=SQuAD2020.0987.0584.07
TASE_IO2019.0986.878.4
QDGATBackbone=RoBERTa2020.0985.8582.74
NumNet+Ensemble=true2020.0985.5982.63
TASE_BIO2019.0985.577.9
TASE_BIO, NO MARG.marginalization=false2019.098576.2
NumNet+Backbone=RoBERTa2020.0984.4281.07
ALBERT-CalculatorBackbone=ALBERT2020.0983.9880.22
TASE_BIO + SSEBackbone=BERT-LARGE2019.0983.976.4
NeRd2019.0981.373.2
MTMSNModel Scale=Large2019.0880.5476.68
SSE2019.0980.270.6
MTMSN2019.0979.969.7
BERT-CALC2019.0978.969.1
MTMSNModel Scale=Base2019.0872.8168.17
NumNetApproach=Numerical MRC2019.1068.3164.92
NumNet2020.0968.3164.92
NABERTModel Scale=Large2019.0867.3564.61
NAQANet+Approach=Numerical MRC2019.1064.8561.47
NABERTModel Scale=Base2019.0858.7555.82
GLaMModel Size=64B/64E, Evaluation Protocol=Few-shot, Shots=22021.1258.6—
GLaMModel Size=64B/64E, Evaluation Protocol=One-shot2021.1257.8—
GLaMModel Size=64B/64E, Evaluation Protocol=Zero-shot2021.1257.3—
NAQANet2019.0849.2446.2
NAQANetApproach=Numerical MRC2019.1049.2446.2
NAQANet2020.0949.2446.2
NAQANetConfiguration=Complete Model2019.0349.2446.2
NAQANetConfiguration=+ Add/Sub2019.0345.7143.07
GPT-3Model Size=175B, Evaluation Protocol=Few-shot, Shots=202021.1236.5—
GPT-3Model Size=175B, Evaluation Protocol=One-shot2021.1234.3—
NAQANetConfiguration=+ Count2019.0333.9230.09
BERTModel Scale=Base2019.0833.3630.1
BERTApproach=Traditional MRC2019.1033.3630.1
BERTBackbone=BERT2020.0933.3630.1
BERTCategory=SQuAD-style RC2019.0333.3630.1
QANetApproach=Traditional MRC2019.1030.4427.5
QANet2020.0930.4427.5
QANetCategory=SQuAD-style RC2019.0330.4427.5
QANet+ELMo2019.0830.3327.71
QANet+ELMoCategory=SQuAD-style RC2019.0330.3327.71
NAQANetConfiguration=+ Q Span2019.0329.1725.94
BiDAF2019.0828.8526.06
BiDAFApproach=Traditional MRC2019.1028.8526.06
BiDAF2020.0928.8526.06
BiDAFCategory=SQuAD-style RC2019.0328.8526.06
GPT-3Model Size=175B, Evaluation Protocol=Zero-shot2021.1223.6—
Semantic Role Labeling2019.0813.6711.03
SRLApproach=Semantic Parsing2019.1011.729.38
SRL2020.0911.729.28
SRLCategory=Semantic Parsing2019.0311.729.28
Syn DepApproach=Semantic Parsing2019.1011.649.38
Syn Dep2020.0911.649.38
Syn DepCategory=Semantic Parsing2019.0311.649.38
OpenIEApproach=Semantic Parsing2019.1011.318.8
OpenIE2020.0911.318.8
OpenIECategory=Semantic Parsing2019.0311.318.8
Heuristic Baseline2019.088.074.28
Q-onlyCategory=Heuristic Baselines2019.038.074.28
P-onlyCategory=Heuristic Baselines2019.032.270.13
MajorityCategory=Heuristic Baselines2019.031.380.09