Constrained Machine Translation on Sand-Glass Zh-De (test)
0.681COMETkiwi Scoregpt-5*
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| gpt-5*Category=w/o Syllable Constraint2026.01 | 0.681 | — | 0.894 | 0.22 | — | |
| claude-4.1-opusCategory=w/o Syllable Constraint2026.01 | 0.676 | — | 0.882 | 0.247 | 24.2 | |
| deepseek-v3Category=w/o Syllable Constraint2026.01 | 0.676 | — | 0.889 | 0.222 | 62.3 | |
| gemini-2.5-pro*Category=w/o Syllable Constraint2026.01 | 0.666 | — | 0.877 | 0.238 | — | |
| claude-4.1-opusCategory=Translate + Modify Pipeline (2 run)2026.01 | 0.653 | — | 0.868 | 0.276 | 42.7 | |
| deepseek-v3Category=Translate + Modify Pipeline (2 run)2026.01 | 0.645 | — | 0.874 | 0.239 | 89.4 | |
| gemini-2.5-pro*Category=Translate + Modify Pipeline (2 run)2026.01 | 0.627 | — | 0.83 | 0.267 | — | |
| claude-4.1-opusCategory=Best of N2026.01 | 0.611 | — | 0.848 | 0.324 | 109.2 | |
| HOMURA (Reason)Category=HOMURA2026.01 | 0.603 | — | 0.861 | 0.321 | 14.7 | |
| HOMURA (Rubric)Category=HOMURA2026.01 | 0.591 | — | 0.853 | 0.317 | 16.3 | |
| gpt-5*Category=Best of N2026.01 | 0.571 | — | 0.861 | 0.275 | — | |
| deepseek-v3Category=w/ Syllable Constraint2026.01 | 0.557 | — | 0.866 | 0.245 | 15.8 | |
| claude-4.1-opusCategory=w/ Syllable Constraint2026.01 | 0.535 | — | 0.838 | 0.293 | 17.2 | |
| gpt-5*Category=w/ Syllable Constraint2026.01 | 0.534 | — | 0.874 | 0.283 | — | |
| deepseek-v3Category=Best of N2026.01 | 0.527 | — | 0.863 | 0.286 | 160.5 | |
| gemini-2.5-pro*Category=w/ Syllable Constraint2026.01 | 0.517 | — | 0.853 | 0.266 | — | |
| gemini-2.5-pro*Category=Best of N2026.01 | 0.517 | — | 0.854 | 0.241 | — | |
| gpt-5*Category=Translate + Modify Pipeline (2 run)2026.01 | 0.514 | — | 0.842 | 0.275 | — |