Extractive Question Answering on SQuAD 2.0
92.9F1 ScoreModernALBERT-large
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ModernALBERT-largeModel Version=large, Parameters=120M2025.12 | 92.9 | — | 85.9 | |
| ModernALBERT-baseModel Version=base, Parameters=75M2025.12 | 92.8 | — | 86.1 | |
| ModernBERT-baseModel Version=base2025.12 | 92.6 | — | 85.2 | |
| ALBERT-xxlargeModel Version=xxlarge2025.12 | 92.5 | — | 84.5 | |
| RoBERTa-baseModel Version=base2025.12 | 91.7 | — | 84.7 | |
| ModernALBERT-mediumModel Version=medium, Parameters=55M2025.12 | 90.4 | — | 82.9 | |
| ModernALBERT-tinyModel Version=tiny, Parameters=50M2025.12 | 90 | — | 82.9 | |
| BERT-baseModel Version=base2025.12 | 88.6 | — | 80.6 | |
| OLTQAConfiguration=Full2023.05 | 79.99 | — | — | |
| ProQA2023.05 | 79.84 | — | — | |
| Hyperformer++2023.05 | 79.52 | — | — | |
| Muppet2023.05 | 79.41 | — | — | |
| OLTQAAblation=without Pk2023.05 | 78.89 | — | — | |
| OLTQAKnowledge Distillation=without MKD2023.05 | 78.81 | — | — | |
| OLTQAKnowledge Distillation=Static MKD2023.05 | 78.73 | — | — | |
| OLTQARetriever=BM252023.05 | 78.49 | — | — | |
| OLTQAKnowledge Distillation=Back KD2023.05 | 78.16 | — | — | |
| OLTQARe-ranker=excluded2023.05 | 77.94 | — | — | |
| UnifiedQA2023.05 | 77.8 | — | — | |
| OLTQAAblation=without Pm2023.05 | 77.72 | — | — | |
| OLTQARetriever=EPR2023.05 | 77.51 | — | — | |
| Rainbow CuckooProtocol=zero-shot, In-domain Transfer=true2025.02 | 69.41 | 73.54 | — | |
| MRQAProtocol=zero-shot, In-domain Transfer=true2025.02 | 66.22 | 66.92 | — | |
| CuckooProtocol=zero-shot2025.02 | 64.06 | 65.26 | — | |
| CuckooProtocol=zero-shot, Training Setting=Only Pre-train2025.02 | 63.36 | 63.94 | — | |
| MetaIEProtocol=zero-shot2025.02 | 62.54 | 55.95 | — | |
| CuckooProtocol=zero-shot, Training Setting=Only Post-train2025.02 | 62.39 | 64.75 | — | |
| Rainbow CuckooProtocol=few-shot2025.02 | 57.67 | 67.36 | — | |
| OPT-C4-TuluV3Protocol=zero-shot2025.02 | 53.81 | 41.54 | — | |
| NuNERProtocol=zero-shot2025.02 | 52.67 | 50.55 | — | |
| MultiNERDProtocol=zero-shot2025.02 | 50.99 | 41.32 | — | |
| CuckooProtocol=few-shot2025.02 | 49.16 | 45.46 | — | |
| ROBERTaProtocol=zero-shot2025.02 | 48.55 | 29.86 | — | |
| EPR2023.05 | 44.14 | — | — |