Speech Translation on CoVoST-2 X-En (test)
43.4De PerformanceAudioPaLM2
Evaluation results
| Method | Links | |||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| AudioPaLM2Architecture Category=End-to-end ST Methods2024.02 | 43.4 | 44.8 | 44.2 | 38.4 | — | 55.6 | 25.5 | 39.1 | 48.3 | 41 | 30 | 7.6 | 48.7 | 35 | 42.6 | 25.9 | 56.2 | |
| GenTranslate-V2Architecture Category=Cascaded ASR+MT Methods2024.02 | 41.1 | 43.1 | 43.3 | 39.5 | — | 55.6 | 24.5 | 38.2 | 44.9 | 37.4 | 27.8 | 10.3 | 48.7 | 30.4 | 42 | 26 | 58.4 | |
| GenTranslate-V2Architecture Category=End-to-end ST Methods2024.02 | 40.6 | 42.7 | 43.6 | 39.4 | — | 54 | 23.3 | 37.7 | 44.8 | 37 | 27.7 | 10.2 | 48 | 30.5 | 42.3 | 25.4 | 55.9 | |
| SeamlessM4T-V2 (ASR+MT)Architecture Category=Cascaded ASR+MT Methods2024.02 | 40.6 | 43 | 43 | 38.8 | — | 55.2 | 22.9 | 36.7 | 43.2 | 33.9 | 27.2 | 8.6 | 47 | 27.8 | 41.9 | 24.7 | 53.1 | |
| GenTranslateArchitecture Category=Cascaded ASR+MT Methods2024.02 | 40.2 | 41.8 | 42.1 | 38.4 | — | 53.7 | 22.9 | 36.9 | 43.8 | 34.3 | 29.4 | 9.5 | 49.7 | 31.2 | 39.6 | 22.3 | 54.6 | |
| SeamlessM4T-Large-V2Architecture Category=End-to-end ST Methods2024.02 | 40 | 42.4 | 42.9 | 39 | — | 53.6 | 22.4 | 36.2 | 42.7 | 33.2 | 26.9 | 8.6 | 46.5 | 27.5 | 41.7 | 23.7 | 52.6 | |
| SeamlessM4T (ASR+MT)Architecture Category=Cascaded ASR+MT Methods2024.02 | 39.8 | 41.5 | 41.1 | 37.5 | — | 53.2 | 21.4 | 35.1 | 42.4 | 29.9 | 26.5 | 8 | 45.2 | 28.8 | 38.6 | 22 | 50.6 | |
| GenTranslateArchitecture Category=End-to-end ST Methods2024.02 | 39.2 | 41.7 | 42 | 38.7 | — | 50.1 | 21.6 | 36.1 | 42.1 | 33.5 | 28.2 | 8.7 | 49.7 | 30.3 | 38.2 | 22.9 | 54.3 | |
| SeamlessM4T-LargeArchitecture Category=End-to-end ST Methods2024.02 | 38.8 | 41.3 | 41.1 | 38.4 | — | 48.6 | 20.9 | 34.5 | 41.1 | 31.2 | 26.3 | 7.5 | 45 | 26.5 | 37.6 | 21.8 | 51.4 | |
| Phi-4-mmLang.=X -> EN2026.04 | 37.9 | — | — | — | — | — | 24.9 | — | — | — | — | — | 5.5 | — | — | 33.3 | — | |
| Whisper + mBART-50Architecture Category=Cascaded ASR+MT Methods2024.02 | 37 | 38.8 | 40.7 | 33 | — | 49 | 21.5 | 30.7 | 39.9 | 32.7 | 16.3 | 0.4 | 37 | 21.4 | 25 | 23 | 45.5 | |
| ComSL-LargeArchitecture Category=End-to-end ST Methods2024.02 | 36 | 38.8 | 40.4 | 35.3 | — | 49.2 | 21.4 | 31.9 | 39.7 | 33.6 | 19.2 | 2.9 | 41.4 | 21.3 | 31.6 | 21.3 | 46.6 | |
| Whisper + NLLB-3.3bArchitecture Category=Cascaded ASR+MT Methods2024.02 | 35.5 | 34.4 | 37.9 | 31.7 | — | 45.4 | 19 | 29.8 | 39.8 | 26.7 | 17.5 | 0.1 | 37 | 20.6 | 29.4 | 25.5 | 45.9 | |
| Whisper-Large V2Architecture Category=End-to-end ST Methods2024.02 | 35 | 35.5 | 39.6 | 31 | — | 42.3 | 16.9 | 28.1 | 40.2 | 27.5 | 14 | 0.2 | 38.5 | 13 | 16.3 | 24.7 | 47.3 | |
| XLS-R-2bArchitecture Category=End-to-end ST Methods2024.02 | 33.6 | 37.6 | 39.2 | 33.8 | — | 39.5 | 9.4 | 22 | 31.7 | 16.7 | 11.1 | 1.6 | 17.1 | 19.5 | 19.6 | 3.5 | 16.5 | |
| AF-Next-instructLang.=X -> EN2026.04 | 33 | — | — | — | — | — | 25.6 | — | — | — | — | — | 29.4 | — | — | 27.2 | — | |
| Cascaded STAux. Data ASR=true, Aux. Data Text=None2023.05 | 21 | 27.6 | 27.4 | 21.3 | 13.5 | 16.8 | 7 | 19.2 | — | — | — | — | — | — | — | — | — | |
| U2TT (LARGE) w/ DUBAux. Data ASR=true, Aux. Data Text=1M2023.05 | 19.5 | 29.5 | 30.9 | 25.2 | 23.9 | 23.2 | 6.1 | 22.6 | — | — | — | — | — | — | — | — | — | |
| Transformer-ST + ASR pre-trainAux. Data ASR=true, Aux. Data Text=None2023.05 | 17.1 | 26.3 | 23 | 18.8 | 11.3 | 14.8 | 5.8 | 16.7 | — | — | — | — | — | — | — | — | — | |
| U2TT (LARGE)Aux. Data ASR=true, Aux. Data Text=None2023.05 | 16.7 | 27.4 | 28.1 | 23.1 | 20 | 21.9 | 5.9 | 20.5 | — | — | — | — | — | — | — | — | — | |
| Revisit STAux. Data ASR=false, Aux. Data Text=None2023.05 | 14.1 | 26.9 | 15.7 | 17.2 | 2.4 | 3.6 | 2 | 11.7 | — | — | — | — | — | — | — | — | — | |
| Transformer-STAux. Data ASR=false, Aux. Data Text=None2023.05 | 8.4 | 4.3 | 12 | 14.4 | 0.2 | 1.2 | 1.4 | 8.8 | — | — | — | — | — | — | — | — | — |