# Label Projection on MLQA (Multi-lingual Metrics) Source: [Label Projection on MLQA (Multi-lingual Metrics)]() Collection index: [llms.txt]() Data updated: 2026-03-13T09:16:02.399Z Evaluation of the model's ability to project labels across multiple languages (Arabic, Chinese, German, Hindi, Spanish, Vietnamese) on the MLQA dataset, using COMET Score and Label Matches F1 metrics. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Label Projection | | Dataset (including any stated split) | MLQA | | Metric | Average COMET Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 32 of 32 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Average COMET Score](): Higher is better; unit: PERCENT. - [Arabic COMET Score](): Higher is better; unit: PERCENT. - [Arabic Label Matches F1](): Higher is better; unit: PERCENT. - [Chinese COMET Score](): Higher is better; unit: PERCENT. - [Chinese Label Matches F1](): Higher is better; unit: PERCENT. - [German COMET Score](): Higher is better; unit: PERCENT. - [German Label Matches F1](): Higher is better; unit: PERCENT. - [Hindi COMET Score](): Higher is better; unit: PERCENT. - [Hindi Label Matches F1](): Higher is better; unit: PERCENT. - [Spanish COMET Score](): Higher is better; unit: PERCENT. - [Spanish Label Matches F1](): Higher is better; unit: PERCENT. - [Vietnamese COMET Score](): Higher is better; unit: PERCENT. - [Vietnamese Label Matches F1](): Higher is better; unit: PERCENT. - [Average Label Matches F1](): Higher is better; unit: PERCENT. ## Recorded results | Method | Average COMET Score | Result date | Report paper | | --- | --- | --- | --- | | Awesome-align | 84.8 | Not recorded | 69b39a2c0636e5bf4afbcc0b | | LabelPigeon | 84.8 | Not recorded | 69b39a2c0636e5bf4afbcc0b | | LabelPigeon | 84.7 | Not recorded | 69b39a2c0636e5bf4afbcc0b | | LabelPigeon | 84.3 | Not recorded | 69b39a2c0636e5bf4afbcc0b | | LabelPigeon | 84 | Not recorded | 69b39a2c0636e5bf4afbcc0b | | LabelPigeon | 84 | Not recorded | 69b39a2c0636e5bf4afbcc0b | | EasyProject | 83.7 | Not recorded | 69b39a2c0636e5bf4afbcc0b | | Awesome-align | 83 | Not recorded | 69b39a2c0636e5bf4afbcc0b | | EasyProject | 82.7 | Not recorded | 69b39a2c0636e5bf4afbcc0b | | Awesome-align | 82.5 | Not recorded | 69b39a2c0636e5bf4afbcc0b | | Awesome-align | 82.4 | Not recorded | 69b39a2c0636e5bf4afbcc0b | | EasyProject | 82.3 | Not recorded | 69b39a2c0636e5bf4afbcc0b | | LabelPigeon | 82.3 | Not recorded | 69b39a2c0636e5bf4afbcc0b | | Awesome-align | 82.2 | Not recorded | 69b39a2c0636e5bf4afbcc0b | | EasyProject | 81.7 | Not recorded | 69b39a2c0636e5bf4afbcc0b | | Awesome-align | 81.6 | Not recorded | 69b39a2c0636e5bf4afbcc0b | | EasyProject | 81.4 | Not recorded | 69b39a2c0636e5bf4afbcc0b | | EasyProject | 80.8 | Not recorded | 69b39a2c0636e5bf4afbcc0b | | Awesome-align | 80.4 | Not recorded | 69b39a2c0636e5bf4afbcc0b | | LabelPigeon | 79.6 | Not recorded | 69b39a2c0636e5bf4afbcc0b | | Gemma 3 27B | 79.3 | Not recorded | 69b39a2c0636e5bf4afbcc0b | | EasyProject | 79.1 | Not recorded | 69b39a2c0636e5bf4afbcc0b | | LabelPigeon | 76.9 | Not recorded | 69b39a2c0636e5bf4afbcc0b | | Awesome-align | 76.7 | Not recorded | 69b39a2c0636e5bf4afbcc0b | | EasyProject | 75.7 | Not recorded | 69b39a2c0636e5bf4afbcc0b | | Gemma 3 27B | 62.2 | Not recorded | 69b39a2c0636e5bf4afbcc0b | | Gemma 3 27B | 59.9 | Not recorded | 69b39a2c0636e5bf4afbcc0b | | Gemma 3 27B | 59.1 | Not recorded | 69b39a2c0636e5bf4afbcc0b | | Gemma 3 27B | 54.6 | Not recorded | 69b39a2c0636e5bf4afbcc0b | | Gemma 3 27B | 53.7 | Not recorded | 69b39a2c0636e5bf4afbcc0b | | Gemma 3 27B | 47.2 | Not recorded | 69b39a2c0636e5bf4afbcc0b | | Gemma 3 27B | 45.3 | Not recorded | 69b39a2c0636e5bf4afbcc0b | ### Awesome-align Average COMET Score: 84.8. Result ID: 69b3d5c047f34ce9249e352b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language":"Arabic"}. This record does not assert independent reproduction. ### LabelPigeon Average COMET Score: 84.8. Result ID: 69b3d5c047f34ce9249e352e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language":"Arabic"}. This record does not assert independent reproduction. ### LabelPigeon Average COMET Score: 84.7. Result ID: 69b3d5c047f34ce9249e3542. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language":"Vietnamese"}. This record does not assert independent reproduction. ### LabelPigeon Average COMET Score: 84.3. Result ID: 69b3d5c047f34ce9249e34fa. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### LabelPigeon Average COMET Score: 84. Result ID: 69b3d5c047f34ce9249e3536. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language":"German"}. This record does not assert independent reproduction. ### LabelPigeon Average COMET Score: 84. Result ID: 69b3d5c047f34ce9249e353e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language":"Spanish"}. This record does not assert independent reproduction. ### EasyProject Average COMET Score: 83.7. Result ID: 69b3d5c047f34ce9249e352d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Language":"Arabic"}. This record does not assert independent reproduction. ### Awesome-align Average COMET Score: 83. Result ID: 69b3d5c047f34ce9249e353f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language":"Vietnamese"}. This record does not assert independent reproduction. ### EasyProject Average COMET Score: 82.7. Result ID: 69b3d5c047f34ce9249e3541. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Language":"Vietnamese"}. This record does not assert independent reproduction. ### Awesome-align Average COMET Score: 82.5. Result ID: 69b3d5c047f34ce9249e353b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language":"Spanish"}. This record does not assert independent reproduction. ### Awesome-align Average COMET Score: 82.4. Result ID: 69b3d5c047f34ce9249e3533. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language":"German"}. This record does not assert independent reproduction. ### EasyProject Average COMET Score: 82.3. Result ID: 69b3d5c047f34ce9249e353d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Language":"Spanish"}. This record does not assert independent reproduction. ### LabelPigeon Average COMET Score: 82.3. Result ID: 69b3d5c047f34ce9249e3546. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language":"Average"}. This record does not assert independent reproduction. ### Awesome-align Average COMET Score: 82.2. Result ID: 69b3d5c047f34ce9249e34f7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### EasyProject Average COMET Score: 81.7. Result ID: 69b3d5c047f34ce9249e34f9. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Awesome-align Average COMET Score: 81.6. Result ID: 69b3d5c047f34ce9249e3543. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language":"Average"}. This record does not assert independent reproduction. ### EasyProject Average COMET Score: 81.4. Result ID: 69b3d5c047f34ce9249e3535. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Language":"German"}. This record does not assert independent reproduction. ### EasyProject Average COMET Score: 80.8. Result ID: 69b3d5c047f34ce9249e3545. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Language":"Average"}. This record does not assert independent reproduction. ### Awesome-align Average COMET Score: 80.4. Result ID: 69b3d5c047f34ce9249e352f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language":"Chinese"}. This record does not assert independent reproduction. ### LabelPigeon Average COMET Score: 79.6. Result ID: 69b3d5c047f34ce9249e3532. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language":"Chinese"}. This record does not assert independent reproduction. ### Gemma 3 27B Average COMET Score: 79.3. Result ID: 69b3d5c047f34ce9249e34f8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Gemma 3","Number of parameters":"27B"}. This record does not assert independent reproduction. ### EasyProject Average COMET Score: 79.1. Result ID: 69b3d5c047f34ce9249e3531. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Language":"Chinese"}. This record does not assert independent reproduction. ### LabelPigeon Average COMET Score: 76.9. Result ID: 69b3d5c047f34ce9249e353a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language":"Hindi"}. This record does not assert independent reproduction. ### Awesome-align Average COMET Score: 76.7. Result ID: 69b3d5c047f34ce9249e3537. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language":"Hindi"}. This record does not assert independent reproduction. ### EasyProject Average COMET Score: 75.7. Result ID: 69b3d5c047f34ce9249e3539. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Language":"Hindi"}. This record does not assert independent reproduction. ### Gemma 3 27B Average COMET Score: 62.2. Result ID: 69b3d5c047f34ce9249e3540. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language":"Vietnamese"}. This record does not assert independent reproduction. ### Gemma 3 27B Average COMET Score: 59.9. Result ID: 69b3d5c047f34ce9249e3534. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language":"German"}. This record does not assert independent reproduction. ### Gemma 3 27B Average COMET Score: 59.1. Result ID: 69b3d5c047f34ce9249e353c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language":"Spanish"}. This record does not assert independent reproduction. ### Gemma 3 27B Average COMET Score: 54.6. Result ID: 69b3d5c047f34ce9249e3544. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language":"Average"}. This record does not assert independent reproduction. ### Gemma 3 27B Average COMET Score: 53.7. Result ID: 69b3d5c047f34ce9249e3530. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language":"Chinese"}. This record does not assert independent reproduction. ### Gemma 3 27B Average COMET Score: 47.2. Result ID: 69b3d5c047f34ce9249e352c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language":"Arabic"}. This record does not assert independent reproduction. ### Gemma 3 27B Average COMET Score: 45.3. Result ID: 69b3d5c047f34ce9249e3538. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Language":"Hindi"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.