Machine Translation Evaluation on WMT MQM 23
95AccRemedy-R
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Remedy-Rθ=32B, TTS/Turns=TTS=62025.12 | 95 | 54.3 | 74.6 | |
| Remedy-Rθ=32B, TTS/Turns=TTS=42025.12 | 94.9 | 54.1 | 74.5 | |
| M-MAD (GPT4o-mini)θ=-, TTS/Turns=3 turns2025.12 | 94.5 | 53.7 | 74.1 | |
| GEMBA-MQM (GPT4)θ=-, TTS/Turns=1 turn2025.12 | 94.5 | 55.2 | 74.9 | |
| Remedy-Rθ=32B, TTS/Turns=1 turn2025.12 | 94.5 | 51.5 | 73 | |
| Remedy-Rθ=32B, TTS/Turns=TTS=32025.12 | 94.5 | 53.9 | 74.2 | |
| Remedy-Rθ=32B, TTS/Turns=TTS=52025.12 | 94.5 | 54.3 | 74.4 | |
| ReMedy9B-23θ=9B, TTS/Turns=12025.12 | 94.1 | 58.2 | 76.2 | |
| Remedy-Rθ=7B, TTS/Turns=TTS=22025.12 | 94.1 | 51.5 | 72.8 | |
| Remedy-Rθ=14B, TTS/Turns=TTS=52025.12 | 94.1 | 55.4 | 74.8 | |
| Remedy-Rθ=14B, TTS/Turns=TTS=62025.12 | 94.1 | 55.7 | 74.9 | |
| Remedy-Rθ=32B, TTS/Turns=TTS=22025.12 | 94.1 | 52.6 | 73.3 | |
| Remedy-Rθ=7B, TTS/Turns=1 turn2025.12 | 93.7 | 49.9 | 71.8 | |
| Remedy-Rθ=7B, TTS/Turns=TTS=32025.12 | 93.7 | 52.6 | 73.1 | |
| Remedy-Rθ=7B, TTS/Turns=TTS=42025.12 | 93.7 | 53.1 | 73.4 | |
| Remedy-Rθ=7B, TTS/Turns=TTS=52025.12 | 93.7 | 53.4 | 73.5 | |
| Remedy-Rθ=7B, TTS/Turns=TTS=62025.12 | 93.7 | 53.6 | 73.7 | |
| Remedy-Rθ=14B, TTS/Turns=TTS=32025.12 | 93.7 | 55.1 | 74.4 | |
| Remedy-Rθ=14B, TTS/Turns=TTS=42025.12 | 93.7 | 55.4 | 74.5 | |
| Remedy-Rθ=14B, TTS/Turns=TTS=22025.12 | 93.3 | 55 | 74.1 | |
| XCOMET-XXLθ=ensemble, TTS/Turns=12025.12 | 92.8 | 57.7 | 75.3 | |
| Remedy-Rθ=14B, TTS/Turns=1 turn2025.12 | 92.4 | 53.6 | 73 | |
| XCOMET-XXL-QEθ=ensemble, TTS/Turns=12025.12 | 91.6 | 55.8 | 73.7 | |
| KIWI-XXLθ=ensemble, TTS/Turns=12025.12 | 91.1 | 54.6 | 72.9 | |
| MetricX-23θ=13B, TTS/Turns=12025.12 | 90.7 | 56.9 | 73.8 | |
| EAPrompt (GPT4o-mini)θ=-, TTS/Turns=1 turn2025.12 | 90.3 | 45.9 | 68.1 | |
| MetricX-23-QEθ=13B, TTS/Turns=12025.12 | 89 | 56.1 | 72.6 |