Machine Translation on WMT 24++ (xCOMET Evaluation)
86.8xCOMET (En->X)Qwen-3.5
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Qwen-3.5Parameter Count=397B-17B2026.06 | 86.8 | — | — | |
| DS-v4-ProParameter Count=1.6T-A49B2026.06 | 85.9 | — | — | |
| DS-v4-FlashParameter Count=284B-A13B2026.06 | 85.9 | — | — | |
| Kimi-K2.6Parameter Count=1T-A32B2026.06 | 84.5 | — | — | |
| GLM-5.1Parameter Count=744B-A40B2026.06 | 84.4 | — | — | |
| Nemotron 3 UltraParameter Count=550B-A55B2026.06 | 83.7 | — | — | |
| MiniMax-2.7Parameter Count=230B-A10B2026.06 | 82.8 | — | — | |
| TranslateGemma-4Bk=–, Expert drop=–, Params=4.3B, Recovery fine-tuning=None, Calibration direction=N/A2026.05 | 0.833 | 0.87 | 0.852 | |
| GPT-OSS-20Bk=–, Expert drop=0%, Params=20.9B, Recovery fine-tuning=None, Calibration direction=N/A2026.05 | 0.799 | 0.862 | 0.83 | |
| Multiling. En→Xk=16, Expert drop=50.00%, Params=11.3B, Recovery fine-tuning=no retrain, Calibration direction=English→X2026.05 | 0.778 | 0.847 | 0.812 | |
| Multiling. En→Xk=22, Expert drop=68.75%, Params=7.8B, Recovery fine-tuning=10k distil., Calibration direction=English→X2026.05 | 0.739 | 0.832 | 0.786 | |
| NLLB-200k=–, Expert drop=–, Params=3.3B, Recovery fine-tuning=None, Calibration direction=N/A2026.05 | 0.734 | 0.833 | 0.784 | |
| Multiling. En→Xk=24, Expert drop=75.00%, Params=6.6B, Recovery fine-tuning=10k distil., Calibration direction=English→X2026.05 | 0.733 | 0.844 | 0.789 | |
| Multiling. En→Xk=20, Expert drop=62.50%, Params=9.0B, Recovery fine-tuning=no retrain, Calibration direction=English→X2026.05 | 0.72 | 0.822 | 0.771 | |
| Multiling. En→Xk=26, Expert drop=81.25%, Params=5.4B, Recovery fine-tuning=10k distil., Calibration direction=English→X2026.05 | 0.706 | 0.839 | 0.772 | |
| Multiling. En→Xk=22, Expert drop=68.75%, Params=7.8B, Recovery fine-tuning=no retrain, Calibration direction=English→X2026.05 | 0.682 | 0.785 | 0.734 | |
| Multiling. En→Xk=28, Expert drop=87.50%, Params=4.2B, Recovery fine-tuning=10k distil., Calibration direction=English→X2026.05 | 0.663 | 0.831 | 0.747 |