Machine Reading Comprehension on SQuAD 1.1 (dev)
89.71EMOurs
Evaluation results
| Method | Links | ||
|---|---|---|---|
| OursBackbone=BERT-base, Strategy=CBBC, Mode=Full Model with Decorrelation2022.03 | 89.71 | 93.18 | |
| XLNetModel Configuration=single model2019.06 | 89.7 | 95.1 | |
| RoBERTaModel Configuration=single model2019.06 | 88.9 | 94.6 | |
| B+C+VcorrBackbone=BERT-base, Strategy=CBBC, Condition=Correlated Capability Scores2022.03 | 87.25 | 91.65 | |
| BERT-large wwmModel Scale=large, Masking Strategy=whole word masking2020.04 | 86.7 | 92.8 | |
| B+CBCLBackbone=BERT-base, Strategy=CBCL2022.03 | 86.15 | 90.89 | |
| SegaBERT-largeModel Scale=large2020.04 | 86 | 92.6 | |
| B+DRCABackbone=BERT-base, Strategy=DRCA2022.03 | 85.05 | 90.59 | |
| B+C+v4Backbone=BERT-base, Strategy=CBBC, Dimension=v42022.03 | 85.03 | 89.9 | |
| B+C+PredictabilityBackbone=BERT-base, Strategy=CBBC, Scoring=Predictability2022.03 | 84.65 | 90.67 | |
| B+CL+VBackbone=BERT-base, Curriculum Strategy=Pre-defined curriculum2022.03 | 84.21 | 90.12 | |
| BERTModel Configuration=single model2019.06 | 84.1 | 90.9 | |
| BERT-largeModel Scale=large2020.04 | 84.1 | 90.9 | |
| B+C+v3Backbone=BERT-base, Strategy=CBBC, Dimension=v32022.03 | 83.66 | 89.68 | |
| B+C+ForgettingBackbone=BERT-base, Strategy=CBBC, Scoring=Forgetting2022.03 | 83.51 | 89.77 | |
| MobileBERT w/o OPT#Params=25.3M2020.04 | 83.4 | 90.3 | |
| SegaBERT-baseModel Scale=base2020.04 | 83.2 | 90.2 | |
| MobileBERT#Params=25.3M2020.04 | 82.9 | 90 | |
| B+C+v2Backbone=BERT-base, Strategy=CBBC, Dimension=v22022.03 | 82.67 | 89.3 | |
| B+C+v1Backbone=BERT-base, Strategy=CBBC, Dimension=v12022.03 | 82.47 | 89.29 | |
| B+C+DatasetMapBackbone=BERT-base, Strategy=CBBC, Scoring=DatasetMap2022.03 | 82.04 | 89.05 | |
| BERT-base¯Model Scale=base, Training=author's pre-training setting2020.04 | 81.9 | 89.4 | |
| MobileBERT TINY#Params=15.1M2020.04 | 81.4 | 88.6 | |
| BBackbone=BERT-base2022.03 | 81.25 | 88.41 | |
| R.M-ReaderModel Type=Ensemble Model2017.05 | 81.2 | 87.9 | |
| BERT BASE#Params=109M2020.04 | 80.8 | 88.5 | |
| BERT-baseModel Scale=base2020.04 | 80.8 | 88.5 | |
| HumanModel Type=Performance Benchmark2017.05 | 80.3 | 90.5 | |
| B+antiCL+VBackbone=BERT-base, Curriculum Strategy=Anti-curriculum2022.03 | 79.8 | 87.31 | |
| BSEModel Type=Ensemble Model2017.05 | 79.6 | 86.6 | |
| DistilBERT BASE-6L#Params=66.6M2020.04 | 79.1 | 86.9 | |
| R.M-ReaderModel Type=Single Model2017.05 | 78.9 | 86.3 | |
| R.M-Reader2018.09 | 78.9 | 86.3 | |
| SANModel Type=Ensemble Model2017.05 | 78.6 | 85.8 | |
| FusionNetModel Type=Ensemble Model2017.05 | 78.5 | 85.8 | |
| DistilBERT BASE-6L‡#Params=66.6M2020.04 | 78.1 | 86.2 | |
| BSEModel Type=Single Model2017.05 | 77.9 | 85.6 | |
| RaSoR+TR+LM2018.09 | 77 | 84 | |
| KAR2018.09 | 76.7 | 84.9 | |
| SANModel Type=Single Model2017.05 | 76.2 | 84.1 | |
| SAN2018.09 | 76.2 | 84.1 | |
| FusionNetModel Type=Single Model2017.05 | 75.3 | 83.6 | |
| FusionNet2018.09 | 75.3 | 83.6 | |
| QANetdata augmentation=true2018.09 | 75.1 | 83.8 | |
| DCN+Model Type=Single Model2017.05 | 74.5 | 83.1 | |
| TinyBERT#Params=14.5M2020.04 | 72.7 | 82.1 | |
| DistilBERT BASE-4L‡#Params=52.2M2020.04 | 71.8 | 81.2 | |
| LR BaselineModel Type=Single Model2017.05 | 40 | 51 |