Machine Translation Meta-evaluation on MENT ZH-EN
80.4Meta ScoreRATE
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| RATEEvaluation Level=System-Level, Metric Category=LLM-as-a-Judge2026.01 | 80.4 | 97.8 | 99.3 | 99.7 | |
| RATEEvaluation Level=Segment-Level, Metric Category=LLM-as-a-Judge2026.01 | 80.4 | 61.9 | 74.5 | 66.4 | |
| GEMBA-DAEvaluation Level=System-Level, Metric Category=LLM-as-a-Judge2026.01 | 77.2 | 86.7 | 96.5 | 87.9 | |
| GEMBA-DAEvaluation Level=Segment-Level, Metric Category=LLM-as-a-Judge2026.01 | 77.2 | 59.1 | 70.1 | 60.2 | |
| COMETEvaluation Level=System-Level, Metric Category=Reference-based Metrics2026.01 | 69 | 88.9 | 98.2 | 91.5 | |
| COMETEvaluation Level=Segment-Level, Metric Category=Reference-based Metrics2026.01 | 69 | 56.5 | 50.3 | 43.3 | |
| GEMBA-MQMEvaluation Level=System-Level, Metric Category=LLM-as-a-Judge2026.01 | 68.7 | 82.2 | 97 | 80.6 | |
| GEMBA-MQMEvaluation Level=Segment-Level, Metric Category=LLM-as-a-Judge2026.01 | 68.7 | 46.2 | 56 | 26.8 | |
| M-MADEvaluation Level=System-Level, Metric Category=LLM-as-a-Judge2026.01 | 65.8 | 82.2 | 96.3 | 83.3 | |
| M-MADEvaluation Level=Segment-Level, Metric Category=LLM-as-a-Judge2026.01 | 65.8 | 43.9 | 39.6 | 23 | |
| ThinMQMEvaluation Level=System-Level, Metric Category=LLM-as-a-Judge2026.01 | 65.1 | 80 | 95.1 | 80.6 | |
| ThinMQMEvaluation Level=Segment-Level, Metric Category=LLM-as-a-Judge2026.01 | 65.1 | 49.5 | 34.8 | 29.8 | |
| BLEUEvaluation Level=System-Level, Metric Category=Reference-based Metrics2026.01 | 64.9 | 82.2 | 91.1 | 84.2 | |
| BLEUEvaluation Level=Segment-Level, Metric Category=Reference-based Metrics2026.01 | 64.9 | 52 | 30.1 | 30.7 | |
| BLEURTEvaluation Level=System-Level, Metric Category=Reference-based Metrics2026.01 | 56.5 | 88.9 | 98.4 | 90.3 | |
| BLEURTEvaluation Level=Segment-Level, Metric Category=Reference-based Metrics2026.01 | 56.5 | 55.4 | 45.4 | 40.5 | |
| MetricX-24Evaluation Level=System-Level, Metric Category=Reference-based Metrics2026.01 | 56.2 | 73.3 | 94.6 | 55.2 | |
| MetricX-24Evaluation Level=Segment-Level, Metric Category=Reference-based Metrics2026.01 | 56.2 | 53.8 | 47.1 | 35.9 | |
| XCOMETEvaluation Level=System-Level, Metric Category=Reference-based Metrics2026.01 | 54.5 | 80 | 96.7 | 73.3 | |
| XCOMETEvaluation Level=Segment-Level, Metric Category=Reference-based Metrics2026.01 | 54.5 | 48.9 | 29.3 | 21.2 | |
| MetricX-23Evaluation Level=System-Level, Metric Category=Reference-based Metrics2026.01 | 52.2 | 73.3 | 94.7 | 62.4 | |
| MetricX-23Evaluation Level=Segment-Level, Metric Category=Reference-based Metrics2026.01 | 52.2 | 47.5 | 31.5 | 17.3 | |
| EAPromptEvaluation Level=System-Level, Metric Category=LLM-as-a-Judge2026.01 | 47.4 | 82.2 | 93.6 | 74.5 | |
| EAPromptEvaluation Level=Segment-Level, Metric Category=LLM-as-a-Judge2026.01 | 47.4 | 36.1 | -6.4 | -10.9 | |
| COMETKiwiEvaluation Level=System-Level, Metric Category=Reference-free (QE) Metrics2026.01 | 42.8 | 51.1 | 90.6 | 5.5 | |
| COMETKiwiEvaluation Level=Segment-Level, Metric Category=Reference-free (QE) Metrics2026.01 | 42.8 | 52.3 | 38.6 | 31.6 | |
| MetricX24-QEEvaluation Level=System-Level, Metric Category=Reference-free (QE) Metrics2026.01 | 41.7 | 48.9 | 88.4 | -1.8 | |
| MetricX24-QEEvaluation Level=Segment-Level, Metric Category=Reference-free (QE) Metrics2026.01 | 41.7 | 47.9 | 31.7 | 18.6 | |
| MetricX23-QEEvaluation Level=System-Level, Metric Category=Reference-free (QE) Metrics2026.01 | 31.9 | 46.7 | 86.4 | -5.5 | |
| MetricX23-QEEvaluation Level=Segment-Level, Metric Category=Reference-free (QE) Metrics2026.01 | 31.9 | 41.9 | 15.2 | 0.4 |