Question Answering on MLQA
80F1 Score (es)BL_BASE
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| BL_BASE2022.12 | 80 | — | — | 2.6 | 1.1 | 3.4 | 1.2 | 1.8 | 1.8 | — | |
| LabelPigeon2026.03 | 72.2 | — | — | 62.7 | 67.5 | 69.7 | 71.5 | 53.4 | 66.1 | — | |
| EasyProject2026.03 | 71.4 | — | — | 62.6 | 65.6 | 70.8 | 72.9 | 53.5 | 66.1 | — | |
| MINIPOSTEFLOPs=29.3, days (V100 GPU days)=11.32022.12 | 66.7 | — | — | 50.8 | 60.4 | 48.9 | 56.6 | 56.7 | 56.7 | — | |
| BL_BASEEFLOPs=54.1, days (V100 GPU days)=20.92022.12 | 66.6 | — | — | 51.2 | 61 | 48.5 | 57.3 | 56.8 | 56.9 | — | |
| MINIJOINTEFLOPs=21.1, days (V100 GPU days)=8.12022.12 | 66 | — | — | 50.8 | 60.1 | 46.4 | 57.2 | 55.6 | 56 | — | |
| BL_SMALLEFLOPs=21.1, days (V100 GPU days)=8.12022.12 | 59.8 | — | — | 46 | 54.6 | 43.1 | 53.2 | 50.8 | 51.2 | — | |
| Forgetting PLMstransfer_mode=Zero-shot cross-lingual transfer2023.07 | 55.3 | — | — | 34.7 | 53.4 | 28.8 | 45 | 43 | 43.4 | 78.3 | |
| MINIPOST2022.12 | 50 | — | — | 1.7 | 0.8 | 1.7 | 0.9 | 1.2 | 1.1 | — | |
| Standard PLMstransfer_mode=Zero-shot cross-lingual transfer2023.07 | 49.4 | — | — | 20.8 | 45.3 | 17.7 | 38.3 | 34.1 | 34.3 | 78.9 | |
| MINIJOINT2022.12 | 40 | — | — | 1 | 0.6 | 5.3 | 0.5 | 0.9 | 1.5 | — | |
| ByT5Scale=Small, Evaluation Protocol=Translate-train2021.05 | — | 67.5 | 49.9 | — | — | — | — | — | — | — | |
| ByT5Scale=Base, Evaluation Protocol=Translate-train2021.05 | — | 71.9 | 54.1 | — | — | — | — | — | — | — | |
| ByT5Scale=Large, Evaluation Protocol=Translate-train2021.05 | — | 74.4 | 56.1 | — | — | — | — | — | — | — | |
| ByT5Scale=XL, Evaluation Protocol=Translate-train2021.05 | — | 75.9 | 57.7 | — | — | — | — | — | — | — | |
| ByT5Scale=XXL, Evaluation Protocol=Translate-train2021.05 | — | 76.9 | 58.8 | — | — | — | — | — | — | — | |
| mT5Scale=Small, Evaluation Protocol=Translate-train2021.05 | — | 56.6 | 38.8 | — | — | — | — | — | — | — | |
| mT5Scale=Base, Evaluation Protocol=Translate-train2021.05 | — | 69.7 | 51 | — | — | — | — | — | — | — | |
| mT5Scale=Large, Evaluation Protocol=Translate-train2021.05 | — | 74 | 55 | — | — | — | — | — | — | — | |
| mT5Scale=XL, Evaluation Protocol=Translate-train2021.05 | — | 75.1 | 56.6 | — | — | — | — | — | — | — | |
| mT5Scale=XXL, Evaluation Protocol=Translate-train2021.05 | — | 76.9 | 58.3 | — | — | — | — | — | — | — |