Metric Correlation with Human Judgments on Hearing-to-Translate five language pairs
0.4743Correlation (aya_canary)COMET
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| COMETmetric variant=ASRwhspr, used texts=src, trg hyp, trg ref2025.11 | 0.4743 | 0.5478 | 0.6621 | 0.5614 | |
| BLEURTused texts=trg hyp, trg ref2025.11 | 0.4736 | 0.5437 | 0.6148 | 0.544 | |
| COMETmetric variant=standard, used texts=src, trg hyp, trg ref2025.11 | 0.4711 | 0.5356 | 0.6581 | 0.5549 | |
| COMETmetric variant=BTmdld, used texts=src, trg hyp, trg ref2025.11 | 0.4655 | 0.5258 | 0.6509 | 0.5474 | |
| COMET-refused texts=trg hyp, trg ref2025.11 | 0.4591 | 0.5319 | 0.6502 | 0.5471 | |
| metricx_qe_normalizedused texts=src, trg hyp2025.11 | 0.4464 | 0.5797 | 0.5578 | 0.528 | |
| BLEUused texts=trg hyp, trg ref2025.11 | 0.3663 | 0.4086 | 0.4714 | 0.4154 | |
| xcomet_qe_by_100used texts=src, trg hyp2025.11 | 0.3439 | 0.5331 | 0.4675 | 0.4482 | |
| ChrFused texts=trg hyp, trg ref2025.11 | 0.2965 | 0.3602 | 0.4237 | 0.3601 | |
| TERused texts=trg hyp, trg ref2025.11 | -0.2984 | -0.2983 | -0.3614 | -0.3194 | |
| MetricX-refused texts=trg hyp, trg ref2025.11 | -0.4409 | -0.4988 | -0.5898 | -0.5098 | |
| metricx_qe_score_strictused texts=src, trg hyp2025.11 | -0.4464 | -0.5797 | -0.5578 | -0.528 | |
| MetricXmetric variant=BTmdld, used texts=src, trg hyp, trg ref2025.11 | -0.4638 | -0.5504 | -0.6117 | -0.542 | |
| MetricXmetric variant=standard, used texts=src, trg hyp, trg ref2025.11 | -0.4882 | -0.5874 | -0.6247 | -0.5668 | |
| MetricXmetric variant=ASRwhspr, used texts=src, trg hyp, trg ref2025.11 | -0.5097 | -0.6102 | -0.6375 | -0.5858 |