Speech-to-Text Translation on FLEURS 9x69 directions (test)
79ARA ScoreMCAT-Large-27B
Evaluation Results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| MCAT-Large-27BModel Category=End-to-end S2TT Models2025.12 | 79 | 80.6 | 86.5 | 83.5 | 80 | 82.6 | 78.2 | 83.2 | 79.9 | 81.5 | |
| Whisper-Large-V3 + NLLB-200-3.3BModel Category=Cascaded ASR+MT Models2025.12 | 78.3 | 79.8 | 83.9 | 81.9 | 79.8 | 81.4 | 78.2 | 82.4 | 78.6 | 80.5 | |
| Whisper-Large-V3 + LLaMAX3-8B-AlpacaModel Category=Cascaded ASR+MT Models2025.12 | 75.3 | 77.8 | 80.8 | 78.8 | 77.5 | 78.4 | 75.6 | 78.2 | 76.3 | 77.6 | |
| SeamlessM4T-V2-LargeModel Category=End-to-end S2TT Models2025.12 | 70.2 | 73.9 | 85.2 | 71.7 | 69.4 | 74 | 69.1 | 72.2 | 72.9 | 73.2 | |
| Whisper-Large-V3 + NLLB-200-3.3BArchitecture=Cascaded ASR+MT2025.12 | 21 | 18 | 30.1 | 23.4 | 18.5 | 19.3 | 16.8 | 23.1 | 18.4 | 21 | |
| MCAT-Large-27BArchitecture=End-to-end S2TT2025.12 | 19 | 16.8 | 30.3 | 23 | 17.1 | 19.7 | 14.8 | 22.8 | 17.1 | 20.1 | |
| SeamlessM4T-V2-LargeArchitecture=End-to-end S2TT2025.12 | 15.8 | 13.5 | 31.7 | 15.4 | 11.9 | 14.3 | 11.7 | 16 | 14.3 | 16.1 | |
| Whisper-Large-V3 + LLaMAX3-8B-AlpacaArchitecture=Cascaded ASR+MT2025.12 | 15.7 | 14.4 | 23.5 | 18.9 | 14.2 | 15.1 | 12.6 | 17.2 | 14.6 | 16.2 |