# Testing the Deliteralization Hypothesis in Human and Machine Translation

Source: [Testing the Deliteralization Hypothesis in Human and Machine Translation](<https://www.sota2.com/research/paper/6a15294adbd7150506f7e1c2>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-27T05:00:21.000Z

Authors: Malik Marmonier, Rachel Bawden, Beno\\^it Sagot

Published: 2026-05-25T10:40:34.000Z

## Abstract

The recent shift from dedicated NMT systems to general-purpose LLMs has reshaped machine translation, with LLMs reported to produce more fluent, less literal output than their predecessors. We test whether this shift extends to the deliteralization hypothesis, the long-standing claim from translation studies that translations become progressively less literal as they are drafted and revised. Using the WMT24++ dataset, we compare the literality of human translations and post-editions to that of two NMT systems and six LLMs across 54 language pairs and three tasks: direct translation, iterative self-revision, and post-editing of human drafts. Literality is measured via a validated Synthetic Literality Index built from six heuristics. We find that (i) human translations remain significantly less literal than those of all tested MT systems, though recent LLMs narrow the gap; (ii) when prompted to iteratively revise their own output, LLMs deliteralize monotonically, providing the first evidence that the hypothesis applies natively to LLM generation; and (iii) as post-editors, LLMs invert the revision triggers of human post-editors, tolerating literal drafts and targeting idiomatic human formulations for revision.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Direct Translation on WMT all 54 LPs 24++](<https://www.sota2.com/research/sota/direct-translation-on-wmt-all-54-lps-24-plus-plus.md>): Direct Translation; WMT all 54 LPs 24++; SLI: not recorded.

- [Iterative Translation Refinement on WMT EN-FR\_FR 24++ (SLI)](<https://www.sota2.com/research/sota/iterative-translation-refinement-on-wmt-en-fr_fr-24-plus-plus-sli.md>): Iterative Translation Refinement; WMT EN-FR\_FR 24++; SLI Iteration 1: not recorded.

- [Post-edition on WMT all 54 LPs 24++](<https://www.sota2.com/research/sota/post-edition-on-wmt-all-54-lps-24-plus-plus.md>): Post-edition; WMT all 54 LPs 24++; SLI: not recorded.

- [Direct Translation on WMT EN-FR\_FR 24++](<https://www.sota2.com/research/sota/direct-translation-on-wmt-en-fr_fr-24-plus-plus.md>): Direct Translation; WMT EN-FR\_FR 24++; SLI: not recorded.

- [Iterative Translation Refinement on WMT all 54 LPs 24++](<https://www.sota2.com/research/sota/iterative-translation-refinement-on-wmt-all-54-lps-24-plus-plus.md>): Iterative Translation Refinement; WMT all 54 LPs 24++; SLI #1: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2605.25686>)

[PDF](<https://arxiv.org/pdf/2605.25686.pdf>)
