# Machine Translation on WMT literary translation (zh→en) 24

Source: [Machine Translation on WMT literary translation (zh→en) 24](<https://www.sota2.com/research/sota/machine-translation-on-wmt-literary-translation-zh-en-24>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-23T10:05:00.801Z

Evaluation of Machine Translation performance on the WMT 2024 Chinese-to-English literary translation dataset, measured using SEGALE\_comet and NA Ratio.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Machine Translation |
| Dataset (including any stated split) | WMT literary translation (zh→en) 24 |
| Metric | SEGALE Comet Score |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 13 of 13 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [SEGALE Comet Score](<https://www.sota2.com/research/sota/machine-translation-on-wmt-literary-translation-zh-en-24.md?metric=SEGALE%20Comet%20Score>): Higher is better; unit: PERCENT.

- [NA Ratio](<https://www.sota2.com/research/sota/machine-translation-on-wmt-literary-translation-zh-en-24.md?metric=NA%20Ratio>): Lower is better; unit: SCALAR.

## Recorded results

| Method | SEGALE Comet Score | Result date | Report paper |
| --- | --- | --- | --- |
| Llama-3.1\_TMPC | 94.62 | Not recorded | 68fa7c241e209131cea8ca8b |
| GPT-4o 2024-08-06 | 94.58 | Not recorded | 68fa7c241e209131cea8ca8b |
| Qwen-2.5 (14B) | 94.43 | Not recorded | 68fa7c241e209131cea8ca8b |
| Llama-3.1\_SFT | 93.54 | Not recorded | 68fa7c241e209131cea8ca8b |
| Llama-3.1\_SimPO | 91.74 | Not recorded | 68fa7c241e209131cea8ca8b |
| Llama-3.1\_Best-of-60 | 90.97 | Not recorded | 68fa7c241e209131cea8ca8b |
| Llama-3.1\_DPO | 90.23 | Not recorded | 68fa7c241e209131cea8ca8b |
| Llama-3.1\_TPO | 88.81 | Not recorded | 68fa7c241e209131cea8ca8b |
| Llama-3.1\_RE-Control | 86.39 | Not recorded | 68fa7c241e209131cea8ca8b |
| Llama-3.1 (8B) | 84.36 | Not recorded | 68fa7c241e209131cea8ca8b |
| Llama-3.1\_ARGS | 63.99 | Not recorded | 68fa7c241e209131cea8ca8b |
| Llama-3.1\_GenARM | 61.18 | Not recorded | 68fa7c241e209131cea8ca8b |
| Llama-3.1\_RAIN | 58.52 | Not recorded | 68fa7c241e209131cea8ca8b |

### Llama-3.1\_TMPC

SEGALE Comet Score: 94.62. Result ID: 69c1104c47f34ce924a9040c.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

Recorded evaluation configuration: {"Test-Time":"✓"}. This record does not assert independent reproduction.

### GPT-4o 2024-08-06

SEGALE Comet Score: 94.58. Result ID: 69c1104c47f34ce924a90400.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa740f1e209131cea5f8b6.md>)

Recorded evaluation configuration: {"Test-Time":"-"}. This record does not assert independent reproduction.

### Qwen-2.5 (14B)

SEGALE Comet Score: 94.43. Result ID: 69c1104c47f34ce924a90401.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa740f1e209131cea5f8b6.md>)

Recorded evaluation configuration: {"Test-Time":"-"}. This record does not assert independent reproduction.

### Llama-3.1\_SFT

SEGALE Comet Score: 93.54. Result ID: 69c1104c47f34ce924a90403.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa66e31e209131cea12d84.md>)

Recorded evaluation configuration: {"Test-Time":"×"}. This record does not assert independent reproduction.

### Llama-3.1\_SimPO

SEGALE Comet Score: 91.74. Result ID: 69c1104c47f34ce924a90404.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75901e209131cea68a5e.md>)

Recorded evaluation configuration: {"Test-Time":"×"}. This record does not assert independent reproduction.

### Llama-3.1\_Best-of-60

SEGALE Comet Score: 90.97. Result ID: 69c1104c47f34ce924a9040b.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

Recorded evaluation configuration: {"Test-Time":"✓"}. This record does not assert independent reproduction.

### Llama-3.1\_DPO

SEGALE Comet Score: 90.23. Result ID: 69c1104c47f34ce924a90405.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74511e209131cea6105d.md>)

Recorded evaluation configuration: {"Test-Time":"×"}. This record does not assert independent reproduction.

### Llama-3.1\_TPO

SEGALE Comet Score: 88.81. Result ID: 69c1104c47f34ce924a9040a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76a81e209131cea6e3d9.md>)

Recorded evaluation configuration: {"Test-Time":"✓"}. This record does not assert independent reproduction.

### Llama-3.1\_RE-Control

SEGALE Comet Score: 86.39. Result ID: 69c1104c47f34ce924a90408.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75901e209131cea68a80.md>)

Recorded evaluation configuration: {"Test-Time":"✓"}. This record does not assert independent reproduction.

### Llama-3.1 (8B)

SEGALE Comet Score: 84.36. Result ID: 69c1104c47f34ce924a90402.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa701f1e209131cea492fe.md>)

[Implementation](<https://github.com/facebookresearch/llama>)

Recorded evaluation configuration: {"Test-Time":"×"}. This record does not assert independent reproduction.

### Llama-3.1\_ARGS

SEGALE Comet Score: 63.99. Result ID: 69c1104c47f34ce924a90406.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa721c1e209131cea5492d.md>)

Recorded evaluation configuration: {"Test-Time":"✓"}. This record does not assert independent reproduction.

### Llama-3.1\_GenARM

SEGALE Comet Score: 61.18. Result ID: 69c1104c47f34ce924a90409.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a0c1e209131cea80cdd.md>)

Recorded evaluation configuration: {"Test-Time":"✓"}. This record does not assert independent reproduction.

### Llama-3.1\_RAIN

SEGALE Comet Score: 58.52. Result ID: 69c1104c47f34ce924a90407.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70c91e209131cea4d1a5.md>)

Recorded evaluation configuration: {"Test-Time":"✓"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
