# Idiomatic Translation on Mixed Dataset en-bn Source: [Idiomatic Translation on Mixed Dataset en-bn]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:48:09.944Z Evaluation of Idiomatic Translation performance using the Mixed Dataset en-bn, measured by LLM-eval and COMET scores. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Idiomatic Translation | | Dataset (including any stated split) | Mixed Dataset en-bn | | Metric | LLM-eval Score | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 18 of 18 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [LLM-eval Score](): Higher is better; unit: SCALAR. - [COMET Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | LLM-eval Score | Result date | Report paper | | --- | --- | --- | --- | | IdiomCE | 2.25 | Not recorded | 68fa78d71e209131cea7ab80 | | Direct | 1.99 | Not recorded | 68fa78d71e209131cea7ab80 | | IdiomCE | 1.783 | Not recorded | 68fa78d71e209131cea7ab80 | | Indictrans2 | 1.78 | Not recorded | 68fa78d71e209131cea7ab80 | | IdiomCE | 1.7 | Not recorded | 68fa78d71e209131cea7ab80 | | NLLB-200 | 1.7 | Not recorded | 68fa78d71e209131cea7ab80 | | Direct | 1.49 | Not recorded | 68fa78d71e209131cea7ab80 | | IdiomCE | 1.46 | Not recorded | 68fa78d71e209131cea7ab80 | | Direct | 1.44 | Not recorded | 68fa78d71e209131cea7ab80 | | Direct | 1.43 | Not recorded | 68fa78d71e209131cea7ab80 | | IdiomCE | 1.4 | Not recorded | 68fa78d71e209131cea7ab80 | | Direct | 1.367 | Not recorded | 68fa78d71e209131cea7ab80 | | Direct | 1.275 | Not recorded | 68fa78d71e209131cea7ab80 | | Direct | 1.23 | Not recorded | 68fa78d71e209131cea7ab80 | | IdiomCE | 1.23 | Not recorded | 68fa78d71e209131cea7ab80 | | IdiomCE | 1.2 | Not recorded | 68fa78d71e209131cea7ab80 | | Direct | 1.17 | Not recorded | 68fa78d71e209131cea7ab80 | | Direct | 1.05 | Not recorded | 68fa78d71e209131cea7ab80 | ### IdiomCE LLM-eval Score: 2.25. Result ID: 6995a6513badfaffe6a3f5a8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-4o"}. This record does not assert independent reproduction. ### Direct LLM-eval Score: 1.99. Result ID: 6995a6513badfaffe6a3f5a9. [Reporting paper]() Recorded evaluation configuration: {"Model":"GPT-4o"}. This record does not assert independent reproduction. ### IdiomCE LLM-eval Score: 1.783. Result ID: 6995a6513badfaffe6a3f5ca. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Gemma2-9b-it"}. This record does not assert independent reproduction. ### Indictrans2 LLM-eval Score: 1.78. Result ID: 6995a6513badfaffe6a3f5c7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Inference Method":"Direct"}. This record does not assert independent reproduction. ### IdiomCE LLM-eval Score: 1.7. Result ID: 6995a6513badfaffe6a3f5a4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Gemma2-9b-it"}. This record does not assert independent reproduction. ### NLLB-200 LLM-eval Score: 1.7. Result ID: 6995a6513badfaffe6a3f5c6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Inference Method":"Direct"}. This record does not assert independent reproduction. ### Direct LLM-eval Score: 1.49. Result ID: 6995a6513badfaffe6a3f5cb. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Gemma2-9b-it"}. This record does not assert independent reproduction. ### IdiomCE LLM-eval Score: 1.46. Result ID: 6995a6513badfaffe6a3f5cc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLama-3.1-8B"}. This record does not assert independent reproduction. ### Direct LLM-eval Score: 1.44. Result ID: 6995a6513badfaffe6a3f5a5. [Reporting paper]() Recorded evaluation configuration: {"Model":"Gemma2-9b-it"}. This record does not assert independent reproduction. ### Direct LLM-eval Score: 1.43. Result ID: 6995a6513badfaffe6a3f5a0. [Reporting paper]() Recorded evaluation configuration: {"Model":"NLLB-200"}. This record does not assert independent reproduction. ### IdiomCE LLM-eval Score: 1.4. Result ID: 6995a6513badfaffe6a3f5a6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLama-3.1-8B"}. This record does not assert independent reproduction. ### Direct LLM-eval Score: 1.367. Result ID: 6995a6513badfaffe6a3f5cd. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"LLama-3.1-8B"}. This record does not assert independent reproduction. ### Direct LLM-eval Score: 1.275. Result ID: 6995a6513badfaffe6a3f5a1. [Reporting paper]() Recorded evaluation configuration: {"Model":"Indictrans2"}. This record does not assert independent reproduction. ### Direct LLM-eval Score: 1.23. Result ID: 6995a6513badfaffe6a3f5a7. [Reporting paper]() Recorded evaluation configuration: {"Model":"LLama-3.1-8B"}. This record does not assert independent reproduction. ### IdiomCE LLM-eval Score: 1.23. Result ID: 6995a6513badfaffe6a3f5c8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLama-3.2-3B"}. This record does not assert independent reproduction. ### IdiomCE LLM-eval Score: 1.2. Result ID: 6995a6513badfaffe6a3f5a2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLama-3.2-3B"}. This record does not assert independent reproduction. ### Direct LLM-eval Score: 1.17. Result ID: 6995a6513badfaffe6a3f5c9. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"LLama-3.2-3B"}. This record does not assert independent reproduction. ### Direct LLM-eval Score: 1.05. Result ID: 6995a6513badfaffe6a3f5a3. [Reporting paper]() Recorded evaluation configuration: {"Model":"LLama-3.2-3B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.