Question Answering on SQuAD 2.0 (test)
89.7EMALBERT
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| ALBERTModel type=Ensemble2019.09 | 89.7 | 92.2 | — | — | — | |
| ELECTRA-1.75MTrain FLOPs=3.1e21 (4.4x), Parameters=335M2020.03 | 88.7 | 91.4 | — | — | — | |
| XLNet + DAAF + VerifierModel type=Ensemble2019.09 | 88.6 | 90.9 | — | — | — | |
| GloSA-sum2026.02 | 88.5 | 91.5 | — | — | — | |
| XLNet + SG-Net VerifierModel type=Ensemble2019.09 | 88.2 | 90.7 | — | — | — | |
| UPMModel type=Ensemble2019.09 | 88.2 | 90.7 | — | — | — | |
| ALBERTModel type=Single model, Training steps=1.5M2019.09 | 88.1 | 90.9 | — | — | — | |
| ALBERTTrain FLOPs=3.1e22 (44x), Parameters=235M2020.03 | 88.1 | 90.9 | — | — | — | |
| ALBERTvariant=Baseline2026.02 | 88.1 | 90.9 | — | — | — | |
| ALBERTvariant=ETC Pipeline2026.02 | 88.1 | 90.97 | — | — | — | |
| ALBERTvariant=ITC Joint2026.02 | 88.1 | 91 | — | — | — | |
| XLNetTrain FLOPs=3.9e21 (5.4x), Parameters=360M2020.03 | 87.9 | 90.7 | — | — | — | |
| UPMModel type=Single model2019.09 | 87.2 | 89.9 | — | — | — | |
| XLNet + SG-Net Verifier++Model type=Single model2019.09 | 87.2 | 90.1 | — | — | — | |
| XLNet + SG-Net VerifierModel Setup=Single model, External training data=true2019.07 | 87 | 89.9 | — | — | — | |
| RoBERTaModel Setup=Single model, External training data=false2019.07 | 86.8 | 89.8 | — | — | — | |
| Human Perf.2019.07 | 86.8 | 89.4 | — | — | — | |
| RoBERTaModel type=Single model2019.09 | 86.8 | 89.8 | — | — | — | |
| RoBERTa-500KTrain FLOPs=3.2e21 (4.5x), Parameters=356M2020.03 | 86.8 | 89.8 | — | — | — | |
| XLNet_LARGEModel Setup=Single model, External training data=true2019.07 | 86.3 | 89.1 | — | — | — | |
| BERT-largeModel type=Single model2019.09 | 86.3 | 89.1 | — | — | — | |
| XLNetModel type=Single model2019.09 | 86.3 | 89.1 | — | — | — | |
| SpanBERT2019.07 | 85.7 | 88.7 | — | — | — | |
| SpanBERTTrain FLOPs=7.1e20 (1x), Parameters=335M2020.03 | 85.7 | 88.7 | — | — | — | |
| Our BERT-1seqtraining=single full-length sequences, NSP=false2019.07 | 83.8 | 86.6 | — | — | — | |
| Our BERTpreprocessing=improved, optimization=improved2019.07 | 82.8 | 85.9 | — | — | — | |
| MPNetSetting=BERT_BASE, Model Type=Single model, Data Augmentation=false2020.04 | 82.8 | 85.8 | — | — | — | |
| GPT-4o-minivariant=Baseline2026.02 | 82 | 85.2 | — | — | — | |
| Google BERT2019.07 | 80 | 83.3 | — | — | — | |
| BERTTrain FLOPs=1.9e20 (0.27x), Parameters=335M2020.03 | 80 | 83 | — | — | — | |
| BERTvariant=Baseline2026.02 | 80 | 83.06 | — | — | — | |
| BERTvariant=ETC Pipeline2026.02 | 80 | 83.23 | — | — | — | |
| BERTvariant=ITC Joint2026.02 | 80 | 83.54 | — | — | — | |
| BERTSetting=BERT_BASE, Model Type=Single model, Data Augmentation=false2020.04 | 73.1 | 76.2 | — | — | — | |
| Baseline (No Retrieval)retrieval_enabled=false2026.01 | — | — | 53.8 | 138 | 0 | |
| L-RAGtau=0.52026.01 | — | — | 78.2 | 227 | 92 | |
| L-RAGtau=12026.01 | — | — | 76 | 209 | 74 | |
| L-RAGtau=1.52026.01 | — | — | 71.6 | 190 | 54 | |
| Oraclecontext=perfect context selection2026.01 | — | — | 80.2 | 235 | — | |
| Standard RAGretrieval_enabled=true2026.01 | — | — | 77.8 | 169 | 100 | |
| Strong RAGretrieval_enabled=true, context=summary + retrieved chunks2026.01 | — | — | 79.8 | 235 | 100 |