# Machine Translation on WMT literary translation (zh→de) 24

Source: [Machine Translation on WMT literary translation (zh→de) 24](<https://www.sota2.com/research/sota/machine-translation-on-wmt-literary-translation-zh-de-24>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-23T10:05:00.862Z

Evaluation of Machine Translation performance on the WMT 2024 Chinese-to-German literary translation dataset, measured using SEGALE\_comet and NA Ratio.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Machine Translation |
| Dataset (including any stated split) | WMT literary translation (zh→de) 24 |
| Metric | SEGALE-COMET Score |
| Unit | SCALAR |
| Direction | Higher is better |
| Results loaded | 13 of 13 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [SEGALE-COMET Score](<https://www.sota2.com/research/sota/machine-translation-on-wmt-literary-translation-zh-de-24.md?metric=SEGALE-COMET%20Score>): Higher is better; unit: SCALAR.

- [NA Ratio](<https://www.sota2.com/research/sota/machine-translation-on-wmt-literary-translation-zh-de-24.md?metric=NA%20Ratio>): Lower is better; unit: PERCENT.

## Recorded results

| Method | SEGALE-COMET Score | Result date | Report paper |
| --- | --- | --- | --- |
| GPT-4o 2024-08-06 | 94.54 | Not recorded | 68fa7c241e209131cea8ca8b |
| Llama-3.1\_DPO | 93.48 | Not recorded | 68fa7c241e209131cea8ca8b |
| Llama-3.1\_SFT | 93.47 | Not recorded | 68fa7c241e209131cea8ca8b |
| Llama-3.1\_SimPO | 93.4 | Not recorded | 68fa7c241e209131cea8ca8b |
| Qwen-2.5 (14B) | 92.98 | Not recorded | 68fa7c241e209131cea8ca8b |
| Llama-3.1\_TMPC | 91.73 | Not recorded | 68fa7c241e209131cea8ca8b |
| Llama-3.1 (8B) | 88.97 | Not recorded | 68fa7c241e209131cea8ca8b |
| Llama-3.1\_TPO | 87.67 | Not recorded | 68fa7c241e209131cea8ca8b |
| Llama-3.1\_RE-Control | 87.16 | Not recorded | 68fa7c241e209131cea8ca8b |
| Llama-3.1\_Best-of-60 | 82.74 | Not recorded | 68fa7c241e209131cea8ca8b |
| Llama-3.1\_RAIN | 67.43 | Not recorded | 68fa7c241e209131cea8ca8b |
| Llama-3.1\_GenARM | 60.96 | Not recorded | 68fa7c241e209131cea8ca8b |
| Llama-3.1\_ARGS | 51.97 | Not recorded | 68fa7c241e209131cea8ca8b |

### GPT-4o 2024-08-06

SEGALE-COMET Score: 94.54. Result ID: 69c1104c47f34ce924a9041a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa740f1e209131cea5f8b6.md>)

Recorded evaluation configuration: {"Test-Time":"-"}. This record does not assert independent reproduction.

### Llama-3.1\_DPO

SEGALE-COMET Score: 93.48. Result ID: 69c1104c47f34ce924a9041f.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74511e209131cea6105d.md>)

Recorded evaluation configuration: {"Test-Time":"×"}. This record does not assert independent reproduction.

### Llama-3.1\_SFT

SEGALE-COMET Score: 93.47. Result ID: 69c1104c47f34ce924a9041d.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa66e31e209131cea12d84.md>)

Recorded evaluation configuration: {"Test-Time":"×"}. This record does not assert independent reproduction.

### Llama-3.1\_SimPO

SEGALE-COMET Score: 93.4. Result ID: 69c1104c47f34ce924a9041e.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75901e209131cea68a5e.md>)

Recorded evaluation configuration: {"Test-Time":"×"}. This record does not assert independent reproduction.

### Qwen-2.5 (14B)

SEGALE-COMET Score: 92.98. Result ID: 69c1104c47f34ce924a9041b.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa740f1e209131cea5f8b6.md>)

Recorded evaluation configuration: {"Test-Time":"-"}. This record does not assert independent reproduction.

### Llama-3.1\_TMPC

SEGALE-COMET Score: 91.73. Result ID: 69c1104c47f34ce924a90426.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

Recorded evaluation configuration: {"Test-Time":"✓"}. This record does not assert independent reproduction.

### Llama-3.1 (8B)

SEGALE-COMET Score: 88.97. Result ID: 69c1104c47f34ce924a9041c.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa701f1e209131cea492fe.md>)

[Implementation](<https://github.com/facebookresearch/llama>)

Recorded evaluation configuration: {"Test-Time":"×"}. This record does not assert independent reproduction.

### Llama-3.1\_TPO

SEGALE-COMET Score: 87.67. Result ID: 69c1104c47f34ce924a90424.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76a81e209131cea6e3d9.md>)

Recorded evaluation configuration: {"Test-Time":"✓"}. This record does not assert independent reproduction.

### Llama-3.1\_RE-Control

SEGALE-COMET Score: 87.16. Result ID: 69c1104c47f34ce924a90422.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75901e209131cea68a80.md>)

Recorded evaluation configuration: {"Test-Time":"✓"}. This record does not assert independent reproduction.

### Llama-3.1\_Best-of-60

SEGALE-COMET Score: 82.74. Result ID: 69c1104c47f34ce924a90425.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

Recorded evaluation configuration: {"Test-Time":"✓"}. This record does not assert independent reproduction.

### Llama-3.1\_RAIN

SEGALE-COMET Score: 67.43. Result ID: 69c1104c47f34ce924a90421.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70c91e209131cea4d1a5.md>)

Recorded evaluation configuration: {"Test-Time":"✓"}. This record does not assert independent reproduction.

### Llama-3.1\_GenARM

SEGALE-COMET Score: 60.96. Result ID: 69c1104c47f34ce924a90423.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a0c1e209131cea80cdd.md>)

Recorded evaluation configuration: {"Test-Time":"✓"}. This record does not assert independent reproduction.

### Llama-3.1\_ARGS

SEGALE-COMET Score: 51.97. Result ID: 69c1104c47f34ce924a90420.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7c241e209131cea8ca8b.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa721c1e209131cea5492d.md>)

Recorded evaluation configuration: {"Test-Time":"✓"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
