# Cross-lingual Question Answering on MLQA v1.0 (test) Source: [Cross-lingual Question Answering on MLQA v1.0 (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:42:53.376Z Evaluation of model performance on the test split of the MLQA v1.0 dataset, focusing on cross-lingual question answering capabilities across multiple languages (English, Spanish, German, Arabic, Hindi, Vietnamese, Chinese). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Cross-lingual Question Answering | | Dataset (including any stated split) | MLQA v1.0 (test) | | Metric | F1 (es) | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 35 of 35 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [F1 (es)](): Higher is better; unit: PERCENT. - [F1 (en)](): Higher is better; unit: PERCENT. - [EM (en)](): Higher is better; unit: PERCENT. - [EM (es)](): Higher is better; unit: PERCENT. - [F1 (de)](): Higher is better; unit: PERCENT. - [EM (de)](): Higher is better; unit: PERCENT. - [F1 (ar)](): Higher is better; unit: PERCENT. - [EM (ar)](): Higher is better; unit: PERCENT. - [F1 (hi)](): Higher is better; unit: PERCENT. - [EM (hi)](): Higher is better; unit: PERCENT. - [F1 (vi)](): Higher is better; unit: PERCENT. - [EM (vi)](): Higher is better; unit: PERCENT. - [F1 (zh)](): Higher is better; unit: PERCENT. - [EM (zh)](): Higher is better; unit: PERCENT. - [Avg F1 (Overall)](): Higher is better; unit: PERCENT. - [Avg EM (Overall)](): Higher is better; unit: PERCENT. ## Recorded results | Method | F1 (es) | Result date | Report paper | | --- | --- | --- | --- | | INFOXLM | 75.1 | Not recorded | 68fa69ae1e209131cea23547 | | XLM-R | 74.7 | Not recorded | 68fa6c2c1e209131cea3289b | | mLUKE-E | 74.5 | Not recorded | 68fa6c2c1e209131cea3289b | | XLM-R | 74.4 | Not recorded | 68fa69ae1e209131cea23547 | | mLUKE-W | 74.3 | Not recorded | 68fa6c2c1e209131cea3289b | | XLM-R | 74.1 | Not recorded | 68fa69ae1e209131cea23547 | | XLM-R | 74.1 | Not recorded | 695b455a53fdd5c361742e54 | | mLUKE-E | 70 | Not recorded | 68fa6c2c1e209131cea3289b | | INFOXLM | 69.8 | Not recorded | 68fa69ae1e209131cea23547 | | XLM-R | 69.8 | Not recorded | 68fa6c2c1e209131cea3289b | | mLUKE-W | 69.7 | Not recorded | 68fa6c2c1e209131cea3289b | | INFOXLM | 69.6 | Not recorded | 68fa69ae1e209131cea23547 | | XLM-K | 69.2 | Not recorded | 68fa6c2c1e209131cea3289b | | UNICODER | 68.6 | Not recorded | 68fa69ae1e209131cea23547 | | XLM | 68 | Not recorded | 68fa69ae1e209131cea23547 | | XLM-15 | 68 | Not recorded | 695b455a53fdd5c361742e54 | | BEPA Monolingual (Ours) | 67.8 | Not recorded | 695b455a53fdd5c361742e54 | | BEPA Bilingual (Ours) | 67.8 | Not recorded | 695b455a53fdd5c361742e54 | | XLM-R | 67.7 | Not recorded | 68fa69ae1e209131cea23547 | | XLM-R | 67.7 | Not recorded | 68fa6c2c1e209131cea3289b | | XLM-R | 67.4 | Not recorded | 68fa69ae1e209131cea23547 | | XLM-R Base | 67.4 | Not recorded | 695b455a53fdd5c361742e54 | | XLM-RoBERTa (Ours) | 66.8 | Not recorded | 695b455a53fdd5c361742e54 | | mBERT | 65.9 | Not recorded | 68fa6c2c1e209131cea3289b | | mBERT | 64.3 | Not recorded | 68fa69ae1e209131cea23547 | | mBERT | 64.3 | Not recorded | 695b455a53fdd5c361742e54 | | GPT-3 (All En) | 24.7 | Not recorded | 68fa6e7d1e209131cea40400 | | GPT-J (All En) | 24.3 | Not recorded | 68fa6e7d1e209131cea40400 | | Toolformer | 20.6 | Not recorded | 68fa6e7d1e209131cea40400 | | Toolformer (disabled) | 19.8 | Not recorded | 68fa6e7d1e209131cea40400 | | GPT-J + CC | 15.7 | Not recorded | 68fa6e7d1e209131cea40400 | | GPT-J | 15.2 | Not recorded | 68fa6e7d1e209131cea40400 | | GPT-3 | 3.4 | Not recorded | 68fa6e7d1e209131cea40400 | | OPT | 0.3 | Not recorded | 68fa6e7d1e209131cea40400 | ### INFOXLM F1 (es): 75.1. Result ID: 698dcff48b517eb43323e632. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"size":"large"}. This record does not assert independent reproduction. ### XLM-R F1 (es): 74.7. Result ID: 698ee84667eb08d78c86757f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Scale":"large"}. This record does not assert independent reproduction. ### mLUKE-E F1 (es): 74.5. Result ID: 698ee84667eb08d78c867581. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Scale":"large","Input Features":"words+entities"}. This record does not assert independent reproduction. ### XLM-R F1 (es): 74.4. Result ID: 698dcff48b517eb43323e631. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"size":"large","reimpl":true}. This record does not assert independent reproduction. ### mLUKE-W F1 (es): 74.3. Result ID: 698ee84667eb08d78c867580. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Scale":"large","Input Features":"words"}. This record does not assert independent reproduction. ### XLM-R F1 (es): 74.1. Result ID: 698dcff48b517eb43323e630. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"size":"large"}. This record does not assert independent reproduction. ### XLM-R F1 (es): 74.1. Result ID: 6996a8195698665ec4f1eed8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Fine-tuned on":"English SQuAD","Evaluation":"Zero-shot"}. This record does not assert independent reproduction. ### mLUKE-E F1 (es): 70. Result ID: 698ee84667eb08d78c86757d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Scale":"base","Input Features":"words+entities"}. This record does not assert independent reproduction. ### INFOXLM F1 (es): 69.8. Result ID: 698dcff48b517eb43323e62e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"size":"base"}. This record does not assert independent reproduction. ### XLM-R F1 (es): 69.8. Result ID: 698ee84667eb08d78c86757b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Scale":"base","Extra Training":"true"}. This record does not assert independent reproduction. ### mLUKE-W F1 (es): 69.7. Result ID: 698ee84667eb08d78c86757c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Scale":"base","Input Features":"words"}. This record does not assert independent reproduction. ### INFOXLM F1 (es): 69.6. Result ID: 698dcff48b517eb43323e62f. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"size":"base","cross-lingual contrast":false}. This record does not assert independent reproduction. ### XLM-K F1 (es): 69.2. Result ID: 698ee84667eb08d78c86757e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Scale":"base"}. This record does not assert independent reproduction. ### UNICODER F1 (es): 68.6. Result ID: 698dcff48b517eb43323e62b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### XLM F1 (es): 68. Result ID: 698dcff48b517eb43323e62a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"source":"Lewis et al., 2020"}. This record does not assert independent reproduction. ### XLM-15 F1 (es): 68. Result ID: 6996a8195698665ec4f1eed6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Fine-tuned on":"English SQuAD","Evaluation":"Zero-shot"}. This record does not assert independent reproduction. ### BEPA Monolingual (Ours) F1 (es): 67.8. Result ID: 6996a8195698665ec4f1eeda. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Fine-tuned on":"English SQuAD","Evaluation":"Zero-shot"}. This record does not assert independent reproduction. ### BEPA Bilingual (Ours) F1 (es): 67.8. Result ID: 6996a8195698665ec4f1eedb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Fine-tuned on":"English SQuAD","Evaluation":"Zero-shot"}. This record does not assert independent reproduction. ### XLM-R F1 (es): 67.7. Result ID: 698dcff48b517eb43323e62d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"size":"base","reimpl":true}. This record does not assert independent reproduction. ### XLM-R F1 (es): 67.7. Result ID: 698ee84667eb08d78c86757a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Scale":"base"}. This record does not assert independent reproduction. ### XLM-R F1 (es): 67.4. Result ID: 698dcff48b517eb43323e62c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"size":"base"}. This record does not assert independent reproduction. ### XLM-R Base F1 (es): 67.4. Result ID: 6996a8195698665ec4f1eed7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Fine-tuned on":"English SQuAD","Evaluation":"Zero-shot"}. This record does not assert independent reproduction. ### XLM-RoBERTa (Ours) F1 (es): 66.8. Result ID: 6996a8195698665ec4f1eed9. [Reporting paper]() Recorded evaluation configuration: {"Fine-tuned on":"English SQuAD","Evaluation":"Zero-shot"}. This record does not assert independent reproduction. ### mBERT F1 (es): 65.9. Result ID: 698ee84667eb08d78c867579. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Scale":"base"}. This record does not assert independent reproduction. ### mBERT F1 (es): 64.3. Result ID: 698dcff48b517eb43323e629. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"source":"Lewis et al., 2020"}. This record does not assert independent reproduction. ### mBERT F1 (es): 64.3. Result ID: 6996a8195698665ec4f1eed5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Fine-tuned on":"English SQuAD","Evaluation":"Zero-shot"}. This record does not assert independent reproduction. ### GPT-3 (All En) F1 (es): 24.7. Result ID: 699025443a6c314849fe6d33. [Reporting paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Contexts and questions in English","Parameters":"175B"}. This record does not assert independent reproduction. ### GPT-J (All En) F1 (es): 24.3. Result ID: 699025443a6c314849fe6d32. [Reporting paper]() Recorded evaluation configuration: {"Evaluation Protocol":"Contexts and questions in English"}. This record does not assert independent reproduction. ### Toolformer F1 (es): 20.6. Result ID: 699025443a6c314849fe6d2f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Tool usage":"enabled"}. This record does not assert independent reproduction. ### Toolformer (disabled) F1 (es): 19.8. Result ID: 699025443a6c314849fe6d2e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Tool usage":"disabled"}. This record does not assert independent reproduction. ### GPT-J + CC F1 (es): 15.7. Result ID: 699025443a6c314849fe6d2d. [Reporting paper]() Recorded evaluation configuration: {"Pre-training":"CCNet"}. This record does not assert independent reproduction. ### GPT-J F1 (es): 15.2. Result ID: 699025443a6c314849fe6d2c. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### GPT-3 F1 (es): 3.4. Result ID: 699025443a6c314849fe6d31. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Parameters":"175B"}. This record does not assert independent reproduction. ### OPT F1 (es): 0.3. Result ID: 699025443a6c314849fe6d30. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Parameters":"66B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.