Machine Reading Comprehension on SQuAD 2.0 (test)
89.6EMRetro-Reader
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Retro-ReaderBackbone=ELECTRA, Verification Strategy=Rear Verification2020.01 | 89.6 | 92.1 | |
| ELECTRAModel=ELECTRA2020.01 | 88.7 | 91.4 | |
| ALBERTModel=ALBERT2020.01 | 88.1 | 90.9 | |
| Retro-ReaderBackbone=ALBERT, Verification Strategy=Rear Verification2020.01 | 88.1 | 91.4 | |
| XLNetModel Configuration=single model2019.06 | 87.926 | 90.689 | |
| XLNet+DSCBackbone=XLNet, Loss Function=DSC (Dice Soft Loss)2019.11 | 87.65 | 89.51 | |
| XLNet+DLBackbone=XLNet, Loss Function=Dice Loss2019.11 | 87.22 | 89.44 | |
| SG-NetModel=SG-Net2020.01 | 87.2 | 90.1 | |
| XLNet+FLBackbone=XLNet, Loss Function=Focal Loss2019.11 | 87.04 | 89.32 | |
| Human2018.08 | 86.9 | 89.5 | |
| RoBERTaModel Configuration=single model2019.06 | 86.82 | 89.795 | |
| HumanTrack=BERT Track2019.08 | 86.8 | 89.5 | |
| HumanModel=Human2020.01 | 86.8 | 89.5 | |
| RoBERTaModel=RoBERTa2020.01 | 86.8 | 89.8 | |
| XLNetModel=XLNet2020.01 | 86.4 | 89.1 | |
| XLNetBackbone=XLNet2019.11 | 86.12 | 88.79 | |
| BERT + DAE + AoATrack=BERT Track, Status=Unpublished2019.08 | 85.9 | 88.6 | |
| BERT + DAE + AoAModel type=Single model2019.09 | 85.9 | 88.6 | |
| BERT + NGM + SSTTrack=BERT Track, Status=Unpublished2019.08 | 85.2 | 87.7 | |
| SG-Net + VerifierTrack=BERT Track, Model Type=Single2019.08 | 85.2 | 87.9 | |
| SG-NetModel type=Single model2019.09 | 85.2 | 87.9 | |
| BERT + NGM + SSTModel type=Single model2019.09 | 85.2 | 87.7 | |
| BERT + CLSTM + MTL + VTrack=BERT Track, Status=Unpublished2019.08 | 84.9 | 88.2 | |
| SemBERTTrack=BERT Track2019.08 | 84.8 | 87.9 | |
| Insight-baseline-BERTTrack=BERT Track, Status=Unpublished2019.08 | 84.8 | 87.6 | |
| SemBERT*_LARGEModel type=Single model, Model variant=Large, Implementation=Our implementation, Synthetic self training=true2019.09 | 84.8 | 87.9 | |
| BERT + MMFT + ADATrack=BERT Track, Status=Unpublished2019.08 | 83 | 85.9 | |
| SemBERT_LARGEModel type=Single model, Model variant=Large, Implementation=Our implementation2019.09 | 82.4 | 85.2 | |
| BERT_LARGETrack=BERT Track2019.08 | 82.1 | 84.8 | |
| BERTModel=BERT2020.01 | 82.1 | 84.8 | |
| NeurQuRIModel=NeurQuRI2020.01 | 81.3 | 84.3 | |
| BERT_LARGEModel type=Single model, Model variant=Large, Implementation=Our implementation2019.09 | 80.5 | 83.6 | |
| BERTModel Configuration=single model2019.06 | 80.005 | 83.061 | |
| BERT+DSCBackbone=BERT, Loss Function=DSC (Dice Soft Loss)2019.11 | 79.02 | 82.95 | |
| BERT+DLBackbone=BERT, Loss Function=Dice Loss2019.11 | 78.99 | 82.88 | |
| BERT+FLBackbone=BERT, Loss Function=Focal Loss2019.11 | 78.92 | 82.2 | |
| BERTBackbone=BERT2019.11 | 78.7 | 81.9 | |
| RMR + ELMo + Verifierembeddings=ELMo, verifier=Model-III2018.08 | 71.7 | 74.2 | |
| RMR + ELMo + VerifierTrack=Regular Track, Model Type=Single2019.08 | 71.7 | 74.2 | |
| RMR + ELMo + VerifierModel type=Single model2019.09 | 71.7 | 74.2 | |
| SLQA+2018.08 | 71.5 | 74.4 | |
| FusionNet++ensemble=true2018.08 | 70.3 | 72.6 | |
| U-NetTrack=Regular Track, Model Type=Single2019.08 | 69.2 | 72.6 | |
| U-NetModel type=Single model2019.09 | 69.2 | 72.6 | |
| Joint SANTrack=Regular Track, Model Type=Single2019.08 | 68.7 | 71.4 | |
| ARRR+2018.08 | 68.6 | 71.1 | |
| SAN2018.08 | 68.6 | 71.4 | |
| VS3-Net+2018.08 | 68.4 | 71.3 | |
| DocQA + ELMoembeddings=ELMo2018.08 | 63.4 | 66.3 | |
| DocQA2018.08 | 59.3 | 62.3 | |
| BNA2018.08 | 59.2 | 62.1 |