# Machine Translation on WMT literary translation (zh→ru) 24

Source: [Machine Translation on WMT literary translation (zh→ru) 24](<https://www.sota2.com/research/sota/machine-translation-on-wmt-literary-translation-zh-ru-24>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-23T10:05:00.833Z

Evaluation of Machine Translation performance on the WMT 2024 Chinese-to-Russian literary translation dataset, measured using SEGALE\_comet and NA Ratio.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Machine Translation |
| Dataset (including any stated split) | WMT literary translation (zh→ru) 24 |
| Metric | SEGALE\_comet |
| Unit | SCALAR |
| Direction | Higher is better |
| Results loaded | 13 of 13 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [SEGALE\_comet](<https://www.sota2.com/research/sota/machine-translation-on-wmt-literary-translation-zh-ru-24.md?metric=SEGALE_comet>): Higher is better; unit: SCALAR.

- [NA Ratio](<https://www.sota2.com/research/sota/machine-translation-on-wmt-literary-translation-zh-ru-24.md?metric=NA%20Ratio>): Lower is better; unit: PERCENT.

## Recorded results

| Method | SEGALE\_comet | Result date | Report paper |
| --- | --- | --- | --- |
| GPT-4o 2024-08-06 | 93.74 | Not recorded | 68fa7c241e209131cea8ca8b |
| Llama-3.1\_TPO | 92.63 | Not recorded | 68fa7c241e209131cea8ca8b |
| Llama-3.1\_TMPC | 91.53 | Not recorded | 68fa7c241e209131cea8ca8b |
| Qwen-2.5 (14B) | 90.47 | Not recorded | 68fa7c241e209131cea8ca8b |
| Llama-3.1\_SFT | 89.11 | Not recorded | 68fa7c241e209131cea8ca8b |
| Llama-3.1 (8B) | 86.28 | Not recorded | 68fa7c241e209131cea8ca8b |
| Llama-3.1\_RE-Control | 84.97 | Not recorded | 68fa7c241e209131cea8ca8b |
| Llama-3.1\_Best-of-60 | 84.86 | Not recorded | 68fa7c241e209131cea8ca8b |
| Llama-3.1\_SimPO | 84.56 | Not recorded | 68fa7c241e209131cea8ca8b |
| Llama-3.1\_DPO | 82.15 | Not recorded | 68fa7c241e209131cea8ca8b |
| Llama-3.1\_RAIN | 66.29 | Not recorded | 68fa7c241e209131cea8ca8b |
| Llama-3.1\_GenARM | 55.67 | Not recorded | 68fa7c241e209131cea8ca8b |
| Llama-3.1\_ARGS | 43.03 | Not recorded | 68fa7c241e209131cea8ca8b |

### GPT-4o 2024-08-06

SEGALE\_comet: 93.74. Result ID: 69c1104c47f34ce924a9040d.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa740f1e209131cea5f8b6.md>)

Recorded evaluation configuration: {"Test-Time":"-"}. This record does not assert independent reproduction.

### Llama-3.1\_TPO

SEGALE\_comet: 92.63. Result ID: 69c1104c47f34ce924a90417.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76a81e209131cea6e3d9.md>)

Recorded evaluation configuration: {"Test-Time":"✓"}. This record does not assert independent reproduction.

### Llama-3.1\_TMPC

SEGALE\_comet: 91.53. Result ID: 69c1104c47f34ce924a90419.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

Recorded evaluation configuration: {"Test-Time":"✓"}. This record does not assert independent reproduction.

### Qwen-2.5 (14B)

SEGALE\_comet: 90.47. Result ID: 69c1104c47f34ce924a9040e.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa740f1e209131cea5f8b6.md>)

Recorded evaluation configuration: {"Test-Time":"-"}. This record does not assert independent reproduction.

### Llama-3.1\_SFT

SEGALE\_comet: 89.11. Result ID: 69c1104c47f34ce924a90410.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa66e31e209131cea12d84.md>)

Recorded evaluation configuration: {"Test-Time":"×"}. This record does not assert independent reproduction.

### Llama-3.1 (8B)

SEGALE\_comet: 86.28. Result ID: 69c1104c47f34ce924a9040f.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa701f1e209131cea492fe.md>)

[Implementation](<https://github.com/facebookresearch/llama>)

Recorded evaluation configuration: {"Test-Time":"×"}. This record does not assert independent reproduction.

### Llama-3.1\_RE-Control

SEGALE\_comet: 84.97. Result ID: 69c1104c47f34ce924a90415.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75901e209131cea68a80.md>)

Recorded evaluation configuration: {"Test-Time":"✓"}. This record does not assert independent reproduction.

### Llama-3.1\_Best-of-60

SEGALE\_comet: 84.86. Result ID: 69c1104c47f34ce924a90418.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

Recorded evaluation configuration: {"Test-Time":"✓"}. This record does not assert independent reproduction.

### Llama-3.1\_SimPO

SEGALE\_comet: 84.56. Result ID: 69c1104c47f34ce924a90411.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75901e209131cea68a5e.md>)

Recorded evaluation configuration: {"Test-Time":"×"}. This record does not assert independent reproduction.

### Llama-3.1\_DPO

SEGALE\_comet: 82.15. Result ID: 69c1104c47f34ce924a90412.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74511e209131cea6105d.md>)

Recorded evaluation configuration: {"Test-Time":"×"}. This record does not assert independent reproduction.

### Llama-3.1\_RAIN

SEGALE\_comet: 66.29. Result ID: 69c1104c47f34ce924a90414.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70c91e209131cea4d1a5.md>)

Recorded evaluation configuration: {"Test-Time":"✓"}. This record does not assert independent reproduction.

### Llama-3.1\_GenARM

SEGALE\_comet: 55.67. Result ID: 69c1104c47f34ce924a90416.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a0c1e209131cea80cdd.md>)

Recorded evaluation configuration: {"Test-Time":"✓"}. This record does not assert independent reproduction.

### Llama-3.1\_ARGS

SEGALE\_comet: 43.03. Result ID: 69c1104c47f34ce924a90413.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa721c1e209131cea5492d.md>)

Recorded evaluation configuration: {"Test-Time":"✓"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
