Machine Reading Comprehension on SQuAD 1.1 (test)
89.898EMXLNet
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| XLNetModel Configuration=single model2019.06 | 89.898 | 95.08 | |
| XLNet+DSCBackbone=XLNet, Loss Function=DSC (Dice Soft Loss)2019.11 | 89.79 | 95.77 | |
| XLNet+DLBackbone=XLNet, Loss Function=Dice Loss2019.11 | 89.13 | 95.36 | |
| XLNetBackbone=XLNet2019.11 | 88.95 | 94.52 | |
| XLNet+FLBackbone=XLNet, Loss Function=Focal Loss2019.11 | 88.9 | 94.55 | |
| BERTModel Configuration=ensemble2019.06 | 87.433 | 93.294 | |
| BERT+DSCBackbone=BERT, Loss Function=DSC (Dice Soft Loss)2019.11 | 85.34 | 91.97 | |
| BERTModel Configuration=single model2019.06 | 85.083 | 91.835 | |
| BERT+DLBackbone=BERT, Loss Function=Dice Loss2019.11 | 84.83 | 91.86 | |
| BERT+FLBackbone=BERT, Loss Function=Focal Loss2019.11 | 84.67 | 91.25 | |
| BERTBackbone=BERT2019.11 | 84.1 | 90.9 | |
| R-net++Model Type=Ensemble Model2017.05 | 82.6 | 88.5 | |
| Hybrid AoA Reader++Model Type=Ensemble Model2017.05 | 82.5 | 89.3 | |
| QANetdata augmentation=true2018.09 | 82.5 | 89.3 | |
| SLQA++Model Type=Ensemble Model2017.05 | 82.4 | 88.6 | |
| R.M-ReaderModel Type=Ensemble Model2017.05 | 82.3 | 88.5 | |
| HumanModel Type=Performance Benchmark2017.05 | 82.3 | 91.2 | |
| AttentionReader++Model Type=Ensemble Model2017.05 | 81.8 | 88.2 | |
| BSEModel Type=Ensemble Model2017.05 | 81 | 87.4 | |
| SLQA++Model Type=Single Model2017.05 | 80.4 | 87 | |
| Hybrid AoA Reader++Model Type=Single Model2017.05 | 80 | 87.3 | |
| R-net++Model Type=Single Model2017.05 | 79.9 | 86.5 | |
| SANModel Type=Ensemble Model2017.05 | 79.6 | 86.5 | |
| R.M-ReaderModel Type=Single Model2017.05 | 79.5 | 86.6 | |
| R.M-Reader2018.09 | 79.5 | 86.6 | |
| FusionNetModel Type=Ensemble Model2017.05 | 79 | 86 | |
| DCN+Model Type=Ensemble Model2017.05 | 78.8 | 86 | |
| BSEModel Type=Single Model2017.05 | 78.6 | 85.8 | |
| RaSoR+TR+LM2018.09 | 77.6 | 84.2 | |
| Pre-trained_T5Backbone=T5-Small (T5_1.0)2022.06 | 77.5 | 86 | |
| AttentionReader++Model Type=Single Model2017.05 | 77.3 | 84.9 | |
| SANModel Type=Single Model2017.05 | 76.8 | 84.4 | |
| SAN2018.09 | 76.8 | 84.4 | |
| KAR2018.09 | 76.1 | 83.5 | |
| FusionNetModel Type=Single Model2017.05 | 76 | 83.9 | |
| FusionNet2018.09 | 76 | 83.9 | |
| DCN+Model Type=Single Model2017.05 | 75.1 | 83.1 | |
| QANetBackbone=QANet2019.11 | 73.6 | 82.7 | |
| Wiki 10KBackbone=T5-Small2022.06 | 67.1 | 76.9 | |
| LIMEBackbone=T5-Small2022.06 | 50.4 | 62.1 | |
| DyckBackbone=T5-Small2022.06 | 50.3 | 62.4 | |
| Nons_SummaryBackbone=T5-Small2022.06 | 48.4 | 60.1 | |
| SetBackbone=T5-Small2022.06 | 48.3 | 60 | |
| LR BaselineModel Type=Single Model2017.05 | 40.4 | 51 | |
| IdentityBackbone=T5-Small2022.06 | 26.1 | 35.4 | |
| Random_InitBackbone=T5-Small2022.06 | 17.2 | 25.2 |