Machine Translation Meta-evaluation on WMT MQM (En-De, En-Es, Ja-Zh) 24
87.9SPARemedy-R-14B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Remedy-R-14BRank=2, Parameter Scale=14B2025.12 | 87.9 | 57.2 | 72.6 | |
| XCOMET-XXLRank=42025.12 | 86.1 | 57.6 | 71.9 | |
| ReMedy9B-24Rank=1, Parameter Scale=9B-242025.12 | 85.9 | 60 | 72.9 | |
| Remedy-R-7BRank=8, Parameter Scale=7B2025.12 | 85.8 | 55.4 | 70.6 | |
| MetricX-24-HybridRank=32025.12 | 85.6 | 58.5 | 72.1 | |
| CometKiwi-XXLtheta=10.5B, reference_anchored=false2026.01 | 85.4 | 55.2 | 70.3 | |
| Remedy-R-32BRank=7, Parameter Scale=32B2025.12 | 85.3 | 56.2 | 70.6 | |
| MetricX-24-Hybrid-QE-XXLtheta=13B, reference_anchored=false2026.01 | 84.9 | 58 | 71.4 | |
| MetricX-24-Hybrid-QERank=52025.12 | 84.9 | 58 | 71.4 | |
| GEMBA-ESAtheta=GPT-4, reference_anchored=false2026.01 | 84.6 | 57.6 | 71.1 | |
| GEMBA-ESA (GPT4)Rank=6, Backbone=GPT-42025.12 | 84.6 | 57.6 | 71.1 | |
| MetricX-24-Hybrid-Largetheta=1.2B, reference_anchored=true2026.01 | 84 | 57 | 70.5 | |
| MetricX-24-Hybrid-QE-XLtheta=3.7B, reference_anchored=false2026.01 | 83.4 | 56.5 | 69.9 | |
| XCOMET-QEtheta=24B, reference_anchored=false, ensemble=true2026.01 | 83.3 | 55.7 | 69.5 | |
| XCOMET-XXL-QERank=92025.12 | 83.3 | 55.7 | 69.5 | |
| Skywork-RMRank=102025.12 | 83.2 | 54.7 | 69 | |
| PEAR-XLref,KDtheta=3.5B, reference_anchored=true, knowledge_distillation=true2026.01 | 82.7 | 58.9 | 70.8 | |
| PEAR-XLboth,KDtheta=3.5B, reference_anchored=false, knowledge_distillation=true2026.01 | 82.7 | 58.3 | 70.5 | |
| PEARref,KDtheta=560M, reference_anchored=true, knowledge_distillation=true2026.01 | 82.6 | 58.7 | 70.7 | |
| COMET-22theta=560M, reference_anchored=true2026.01 | 82.4 | 55.4 | 68.9 | |
| BLEURT-20theta=579M, reference_anchored=true2026.01 | 82.1 | 55 | 68.6 | |
| PEARboth,KDtheta=560M, reference_anchored=false, knowledge_distillation=true2026.01 | 82 | 58.8 | 70.4 | |
| PEAR-XLreftheta=3.5B, reference_anchored=true, knowledge_distillation=false2026.01 | 81.6 | 59.2 | 70.4 | |
| PEAR-XLboththeta=3.5B, reference_anchored=false, knowledge_distillation=false2026.01 | 81.6 | 58.8 | 70.2 | |
| PEARreftheta=560M, reference_anchored=true, knowledge_distillation=false2026.01 | 81.5 | 59 | 70.3 | |
| PEARboththeta=560M, reference_anchored=false, knowledge_distillation=false2026.01 | 81.2 | 59 | 70.1 | |
| MetricX-24-Hybrid-QE-Largetheta=1.2B, reference_anchored=false2026.01 | 80.6 | 56.1 | 68.3 | |
| CometKiwitheta=560M, reference_anchored=false2026.01 | 73.3 | 54.7 | 64 |