# Machine Translation Quality Estimation (Completeness) on WMT DA 2022 (test) Source: [Machine Translation Quality Estimation (Completeness) on WMT DA 2022 (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:39:38.323Z Evaluation of Machine Translation Quality Estimation performance, specifically focusing on the Completeness aspect, using the WMT DA 2022 test set across various language pairs. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Machine Translation Quality Estimation (Completeness) | | Dataset (including any stated split) | WMT DA 2022 (test) | | Metric | Completeness Score (AR) | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Completeness Score (AR)](): Higher is better; unit: PERCENT. - [Completeness Score (ES)](): Higher is better; unit: PERCENT. - [Completeness Score (HE)](): Higher is better; unit: PERCENT. - [Completeness Score (JA)](): Higher is better; unit: PERCENT. - [Completeness Score (TR)](): Higher is better; unit: PERCENT. - [Completeness Score (UKR)](): Higher is better; unit: PERCENT. - [Completeness Score (YO)](): Higher is better; unit: PERCENT. - [Completeness Score (ZH)](): Higher is better; unit: PERCENT. ## Recorded results | Method | Completeness Score (AR) | Result date | Report paper | | --- | --- | --- | --- | | STEEREVAL + COMET (wmt22-da) | 33 | Not recorded | 69730077b2be0ed85a78c981 | | STEEREVAL + COMET (wmt22-da) | 29 | Not recorded | 69730077b2be0ed85a78c981 | | STEEREVAL + GPTScore (Aya-exp 32b) | 27 | Not recorded | 69730077b2be0ed85a78c981 | | STEEREVAL + GPTScore (Aya-exp 32b) | 26 | Not recorded | 69730077b2be0ed85a78c981 | | STEEREVAL + Direct Prompting (Llama3-8b) | 25 | Not recorded | 69730077b2be0ed85a78c981 | | STEEREVAL + Direct Prompting (Llama3-8b) | 23 | Not recorded | 69730077b2be0ed85a78c981 | | STEEREVAL + GPTScore (Aya-exp 8b) | 23 | Not recorded | 69730077b2be0ed85a78c981 | | STEEREVAL + GPTScore (Aya-exp 8b) | 23 | Not recorded | 69730077b2be0ed85a78c981 | | STEEREVAL + Direct Prompting (Aya-exp 32b) | 21 | Not recorded | 69730077b2be0ed85a78c981 | | STEEREVAL + Direct Prompting (Aya-exp 32b) | 21 | Not recorded | 69730077b2be0ed85a78c981 | | STEEREVAL + GPTScore (Bloom-7b) | 20 | Not recorded | 69730077b2be0ed85a78c981 | | STEEREVAL + GPTScore (Bloom-7b) | 19 | Not recorded | 69730077b2be0ed85a78c981 | | STEEREVAL + GPTScore (Llama3-8b) | 15 | Not recorded | 69730077b2be0ed85a78c981 | | STEEREVAL + GPTScore (Llama3-8b) | 15 | Not recorded | 69730077b2be0ed85a78c981 | | STEEREVAL + Direct Prompting (Aya-exp 8b) | 9 | Not recorded | 69730077b2be0ed85a78c981 | | STEEREVAL + Direct Prompting (Aya-exp 8b) | 8 | Not recorded | 69730077b2be0ed85a78c981 | | STEEREVAL + Direct Prompting (Bloom-7b) | -12 | Not recorded | 69730077b2be0ed85a78c981 | | STEEREVAL + Direct Prompting (Bloom-7b) | -14 | Not recorded | 69730077b2be0ed85a78c981 | ### STEEREVAL + COMET (wmt22-da) Completeness Score (AR): 33. Result ID: 699674d35698665ec4bc368f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Framework":"COMET","Base Model":"wmt22-da","Representation":"Map"}. This record does not assert independent reproduction. ### STEEREVAL + COMET (wmt22-da) Completeness Score (AR): 29. Result ID: 699674d35698665ec4bc368e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Framework":"COMET","Base Model":"wmt22-da","Representation":"Vector"}. This record does not assert independent reproduction. ### STEEREVAL + GPTScore (Aya-exp 32b) Completeness Score (AR): 27. Result ID: 699674d35698665ec4bc369e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Framework":"GPTScore","Base Model":"Aya-exp 32b","Representation":"Vector"}. This record does not assert independent reproduction. ### STEEREVAL + GPTScore (Aya-exp 32b) Completeness Score (AR): 26. Result ID: 699674d35698665ec4bc369f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Framework":"GPTScore","Base Model":"Aya-exp 32b","Representation":"Map"}. This record does not assert independent reproduction. ### STEEREVAL + Direct Prompting (Llama3-8b) Completeness Score (AR): 25. Result ID: 699674d35698665ec4bc3693. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Framework":"Direct Prompting","Base Model":"Llama3-8b","Representation":"Map"}. This record does not assert independent reproduction. ### STEEREVAL + Direct Prompting (Llama3-8b) Completeness Score (AR): 23. Result ID: 699674d35698665ec4bc3692. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Framework":"Direct Prompting","Base Model":"Llama3-8b","Representation":"Vector"}. This record does not assert independent reproduction. ### STEEREVAL + GPTScore (Aya-exp 8b) Completeness Score (AR): 23. Result ID: 699674d35698665ec4bc369c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Framework":"GPTScore","Base Model":"Aya-exp 8b","Representation":"Vector"}. This record does not assert independent reproduction. ### STEEREVAL + GPTScore (Aya-exp 8b) Completeness Score (AR): 23. Result ID: 699674d35698665ec4bc369d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Framework":"GPTScore","Base Model":"Aya-exp 8b","Representation":"Map"}. This record does not assert independent reproduction. ### STEEREVAL + Direct Prompting (Aya-exp 32b) Completeness Score (AR): 21. Result ID: 699674d35698665ec4bc3696. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Framework":"Direct Prompting","Base Model":"Aya-exp 32b","Representation":"Vector"}. This record does not assert independent reproduction. ### STEEREVAL + Direct Prompting (Aya-exp 32b) Completeness Score (AR): 21. Result ID: 699674d35698665ec4bc3697. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Framework":"Direct Prompting","Base Model":"Aya-exp 32b","Representation":"Map"}. This record does not assert independent reproduction. ### STEEREVAL + GPTScore (Bloom-7b) Completeness Score (AR): 20. Result ID: 699674d35698665ec4bc3698. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Framework":"GPTScore","Base Model":"Bloom-7b","Representation":"Vector"}. This record does not assert independent reproduction. ### STEEREVAL + GPTScore (Bloom-7b) Completeness Score (AR): 19. Result ID: 699674d35698665ec4bc3699. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Framework":"GPTScore","Base Model":"Bloom-7b","Representation":"Map"}. This record does not assert independent reproduction. ### STEEREVAL + GPTScore (Llama3-8b) Completeness Score (AR): 15. Result ID: 699674d35698665ec4bc369a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Framework":"GPTScore","Base Model":"Llama3-8b","Representation":"Vector"}. This record does not assert independent reproduction. ### STEEREVAL + GPTScore (Llama3-8b) Completeness Score (AR): 15. Result ID: 699674d35698665ec4bc369b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Framework":"GPTScore","Base Model":"Llama3-8b","Representation":"Map"}. This record does not assert independent reproduction. ### STEEREVAL + Direct Prompting (Aya-exp 8b) Completeness Score (AR): 9. Result ID: 699674d35698665ec4bc3694. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Framework":"Direct Prompting","Base Model":"Aya-exp 8b","Representation":"Vector"}. This record does not assert independent reproduction. ### STEEREVAL + Direct Prompting (Aya-exp 8b) Completeness Score (AR): 8. Result ID: 699674d35698665ec4bc3695. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Framework":"Direct Prompting","Base Model":"Aya-exp 8b","Representation":"Map"}. This record does not assert independent reproduction. ### STEEREVAL + Direct Prompting (Bloom-7b) Completeness Score (AR): -12. Result ID: 699674d35698665ec4bc3690. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Framework":"Direct Prompting","Base Model":"Bloom-7b","Representation":"Vector"}. This record does not assert independent reproduction. ### STEEREVAL + Direct Prompting (Bloom-7b) Completeness Score (AR): -14. Result ID: 699674d35698665ec4bc3691. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Framework":"Direct Prompting","Base Model":"Bloom-7b","Representation":"Map"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.