Speech-to-Text Translation on FLEURS (test)
86.3Avg. ScoreMCAT-Large-27B
Evaluation results
| Method | Links | ||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MCAT-Large-27Bparameters=27B2025.12 | 86.3 | 86.1 | 85.2 | 90.1 | 87.2 | 86.5 | 86.9 | 86.3 | 87.2 | 78.9 | 90.2 | 86.8 | 90.4 | 79.7 | 88.5 | 83.1 | 87.5 | 85 | 86.6 | 88.1 | 87.8 | 88.8 | 85.2 | 83.3 | 87.7 | 88.2 | 80.9 | 87.8 | |
| MCAT-Small-9Bparameters=9B2025.12 | 85.9 | 85.4 | 85 | 89.4 | 87 | 85.7 | 86.4 | 85.6 | 86.6 | 78.1 | 89.2 | 86 | 89.8 | 81.4 | 88.3 | 82.6 | 87.1 | 87 | 86.2 | 87.6 | 87.1 | 87.7 | 84.6 | 82.7 | 87.1 | 87.6 | 80.8 | 87.1 | |
| Qwen3-Omni-30B-A3B-Instructparameters=30B, variant=A3B-Instruct2025.12 | 85 | 86.6 | 83.3 | 89 | 88.3 | 86.8 | 85.5 | 87.3 | 73.9 | 77.7 | 90.9 | 87.3 | 90.8 | 77.6 | 89.7 | 80.2 | 88.3 | 71.9 | 86.2 | 87.3 | 88.4 | 88.9 | 85.4 | 80.1 | 88.8 | 88.1 | 78.3 | 88.5 | |
| SeamlessM4T-V2-Largemodel_size=Large2025.12 | 84.3 | 84.5 | 84.6 | 88 | 79.7 | 84.8 | 84.6 | 85.3 | 84.5 | 78.1 | 89 | 85.1 | 84.7 | 79.9 | 85.1 | 81.4 | 86.6 | 85.7 | 85.1 | 85.7 | 86.6 | 86.3 | 83.2 | 82.1 | 83.3 | 86.7 | 79.4 | 85.6 | |
| Qwen2.5-Omni-7Bparameters=7B2025.12 | 73.8 | 84.5 | 63 | 81.9 | 86.5 | 85 | 73.4 | 84.6 | 66.1 | 58.7 | 86.2 | 84.2 | 88.3 | 37.8 | 84.7 | 38.7 | 83.5 | 41.1 | 82.1 | 81.5 | 86.7 | 85.2 | 83.4 | 55.6 | 82.1 | 79.2 | 51.6 | 76.4 | |
| VIRBABackbone=Qwen2.5-Omni2025.08 | 36.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Qwen2.5-Omni2025.08 | 35.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Kimi-Audio2025.08 | 35.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Step-Audio-R12025.08 | 34.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AF32025.08 | 34.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| VIRBABackbone=Qwen2-Audio2025.08 | 34 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Group-DPO2025.08 | 33.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SeamlessM4T-L V22025.08 | 29.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Qwen2-Audiovariant=base2025.08 | 28.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Whisper-large V22025.08 | 23.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |