Constrained Machine Translation on Sand-Glass Zh-En (test)
0.748Cometkiwideepseek-v3
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| deepseek-v3Category=w/o Syllable Constraint2026.01 | 0.748 | — | 0.941 | 0.287 | 46.3 | |
| gpt-5*Category=w/o Syllable Constraint2026.01 | 0.746 | — | 0.947 | 0.3 | — | |
| claude-4.1-opusCategory=w/o Syllable Constraint2026.01 | 0.741 | — | 0.943 | 0.313 | 17.1 | |
| gemini-2.5-pro*Category=w/o Syllable Constraint2026.01 | 0.739 | — | 0.941 | 0.307 | — | |
| gpt-5*Category=w/ Syllable Constraint2026.01 | 0.705 | — | 0.928 | 0.345 | — | |
| deepseek-v3Category=Translate + Modify Pipeline (2 run)2026.01 | 0.703 | — | 0.934 | 0.322 | 88.1 | |
| HOMURA (Reason)Category=HOMURA2026.01 | 0.701 | — | 0.925 | 0.376 | 10.1 | |
| deepseek-v3Category=w/ Syllable Constraint2026.01 | 0.7 | — | 0.924 | 0.307 | 12.4 | |
| HOMURA (Rubric)Category=HOMURA2026.01 | 0.7 | — | 0.914 | 0.378 | 9.2 | |
| gemini-2.5-pro*Category=w/ Syllable Constraint2026.01 | 0.698 | — | 0.927 | 0.322 | — | |
| claude-4.1-opusCategory=Translate + Modify Pipeline (2 run)2026.01 | 0.694 | — | 0.904 | 0.327 | 30.2 | |
| claude-4.1-opusCategory=w/ Syllable Constraint2026.01 | 0.691 | — | 0.907 | 0.376 | 12.2 | |
| claude-4.1-opusCategory=Best of N2026.01 | 0.691 | — | 0.903 | 0.367 | 109.7 | |
| deepseek-v3Category=Best of N2026.01 | 0.691 | — | 0.902 | 0.332 | 122.6 | |
| gemini-2.5-pro*Category=Translate + Modify Pipeline (2 run)2026.01 | 0.691 | — | 0.902 | 0.348 | — | |
| gpt-5*Category=Translate + Modify Pipeline (2 run)2026.01 | 0.69 | — | 0.917 | 0.321 | — | |
| gemini-2.5-pro*Category=Best of N2026.01 | 0.673 | — | 0.904 | 0.276 | — | |
| gpt-5*Category=Best of N2026.01 | 0.673 | — | 0.906 | 0.296 | — |