Speech Translation on MuST-C (test)
31.54BLEU Scorecontr-cos-all + asr
Evaluation results
| Method | Links | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| contr-cos-all + asrFT Data=100%, Giga Dataset=true2024.12 | 31.54 | — | — | — | — | — | — | — | — | — | — | — | — | — | 81.99 | — | — | — | |
| contr-cos-allFT Data=100%, Giga Dataset=true2024.12 | 31.36 | — | — | — | — | — | — | — | — | — | — | — | — | — | 81.98 | — | — | — | |
| contr-wasser-all + asrFT Data=100%, Giga Dataset=true2024.12 | 31.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | 82 | — | — | — | |
| SpecializedFT Data=N/A, Giga Dataset=false2024.12 | 30.99 | — | — | — | — | — | — | — | — | — | — | — | — | — | 80.02 | — | — | — | |
| contr-wasser-allFT Data=100%, Giga Dataset=true2024.12 | 30.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | 81.65 | — | — | — | |
| contr-cos-all + asrFT Data=100%, Giga Dataset=false2024.12 | 30.82 | — | — | — | — | — | — | — | — | — | — | — | — | — | 81.56 | — | — | — | |
| contr-wasser-all + asrFT Data=100%, Giga Dataset=false2024.12 | 30.75 | — | — | — | — | — | — | — | — | — | — | — | — | — | 81.62 | — | — | — | |
| contr-cos-allFT Data=100%, Giga Dataset=false2024.12 | 30.69 | — | — | — | — | — | — | — | — | — | — | — | — | — | 81.48 | — | — | — | |
| contr-wasser-allFT Data=100%, Giga Dataset=false2024.12 | 30.62 | — | — | — | — | — | — | — | — | — | — | — | — | — | 81.59 | — | — | — | |
| ASR pretrainFT Data=100%, Giga Dataset=true2024.12 | 30.46 | — | — | — | — | — | — | — | — | — | — | — | — | — | 81.03 | — | — | — | |
| ASR pretrainFT Data=100%, Giga Dataset=false2024.12 | 30.31 | — | — | — | — | — | — | — | — | — | — | — | — | — | 80.98 | — | — | — | |
| contr-wasser-all + asrFT Data=10%, Giga Dataset=true2024.12 | 30.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | 81.45 | — | — | — | |
| contr-cos-allFT Data=10%, Giga Dataset=true2024.12 | 29.95 | — | — | — | — | — | — | — | — | — | — | — | — | — | 81.21 | — | — | — | |
| contr-cos-all + asrFT Data=10%, Giga Dataset=true2024.12 | 29.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | 81.29 | — | — | — | |
| contr-wasser-allFT Data=10%, Giga Dataset=true2024.12 | 29.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | 80.79 | — | — | — | |
| contr-wasser-allFT Data=10%, Giga Dataset=false2024.12 | 29.07 | — | — | — | — | — | — | — | — | — | — | — | — | — | 80.52 | — | — | — | |
| contr-wasser-all + asrFT Data=10%, Giga Dataset=false2024.12 | 29.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | 80.26 | — | — | — | |
| contr-cos-embFT Data=10%, Giga Dataset=true2024.12 | 28.76 | — | — | — | — | — | — | — | — | — | — | — | — | — | 80.18 | — | — | — | |
| BLSP-lslm-7bFT Data=N/A, Giga Dataset=false2024.12 | 28.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | 78.68 | — | — | — | |
| contr-cos-all + asrFT Data=10%, Giga Dataset=false2024.12 | 27.61 | — | — | — | — | — | — | — | — | — | — | — | — | — | 78.94 | — | — | — | |
| ASR pretrainFT Data=10%, Giga Dataset=true2024.12 | 27.44 | — | — | — | — | — | — | — | — | — | — | — | — | — | 78.38 | — | — | — | |
| contr-cos-allFT Data=10%, Giga Dataset=false2024.12 | 27.19 | — | — | — | — | — | — | — | — | — | — | — | — | — | 78.29 | — | — | — | |
| contr-wasser-embFT Data=10%, Giga Dataset=true2024.12 | 26.47 | — | — | — | — | — | — | — | — | — | — | — | — | — | 77.59 | — | — | — | |
| contr-wasser-embFT Data=10%, Giga Dataset=false2024.12 | 26.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | 77.09 | — | — | — | |
| contr-cos-embFT Data=10%, Giga Dataset=false2024.12 | 25.47 | — | — | — | — | — | — | — | — | — | — | — | — | — | 76.32 | — | — | — | |
| ASR pretrainFT Data=10%, Giga Dataset=false2024.12 | 24.82 | — | — | — | — | — | — | — | — | — | — | — | — | — | 75.7 | — | — | — | |
| Qwen2-Audio-7bFT Data=N/A, Giga Dataset=false2024.12 | 21.57 | — | — | — | — | — | — | — | — | — | — | — | — | — | 74.81 | — | — | — | |
| SLAM-LLM (Ours-7B)Architecture Category=End-to-End, Zero-shot=true2026.01 | 16.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 18.3 | 11.3 | 21.2 | |
| Qwen2-AudioArchitecture Category=End-to-End, Zero-shot=true2026.01 | 16.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 22.4 | 7.8 | 18.3 | |
| Baselinetrainable/total parameters (M)=76.3/76.3, group=Ours2021.06 | — | 24.18 | 28.28 | 34.98 | 24.62 | 28.8 | 31.13 | 23.22 | 15.88 | 26.39 | — | — | — | — | — | — | — | — | |
| Best adaptingtrainable/total parameters (M)=8 x 4.8 / 76.3, group=Ours2021.06 | — | 24.63 | 28.73 | 34.75 | 24.96 | 28.8 | 30.96 | 23.7 | 16.36 | 26.61 | — | — | — | — | — | — | — | — | |
| Best fine-tuningtrainable/total parameters (M)=8 x 35.5 / 8 x 76.3, group=Ours2021.06 | — | 24.5 | 28.67 | 34.89 | 24.82 | 28.38 | 30.73 | 23.78 | 16.23 | 26.5 | — | — | — | — | — | — | — | — | |
| BLSP-KDArchitecture Category=End-to-End, Zero-shot=true2026.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 20.7 | — | — | |
| ConformerArchitecture=AR, Reference=Inaguma et al., 2021a, Seq-KD=false2023.05 | — | 25 | 30.5 | 35.5 | 25.4 | 29.7 | — | — | — | — | — | — | — | — | — | — | — | — | |
| ConformerArchitecture=AR, Reference=Inaguma et al., 2021a, Seq-KD=true2023.05 | — | 26.3 | 31 | 36.4 | 25.9 | 30.6 | — | — | — | — | — | — | — | — | — | — | — | — | |
| ConSTArchitecture=AR2023.05 | — | 25.7 | 30.4 | 36.8 | 26.3 | 30.6 | 32 | 24.8 | 17.3 | 28 | — | — | — | — | — | — | — | — | |
| CTCArchitecture=NAR, Reference=Inaguma et al., 2021b2023.05 | — | 19.4 | — | 27.4 | — | — | — | — | — | — | — | — | — | 20.84 | — | — | — | — | |
| CTCArchitecture=NAR, Reference=Inaguma et al., 2021a2023.05 | — | 24.1 | 29 | 34.6 | 24.3 | 28.5 | — | — | — | — | — | — | — | 13.83 | — | — | — | — | |
| CTC-Aug STArchitecture=AR, Seq-KD=false2023.05 | — | 26.9 | 31.5 | 38.1 | 27.4 | 31.9 | 33.4 | 25.8 | 18.7 | 29.2 | — | — | 16.1 | 1 | — | — | — | — | |
| CTC-Aug STArchitecture=AR, Seq-KD=true2023.05 | — | 27.7 | 31.6 | 39.5 | 27.5 | 32.3 | 33.7 | 26.6 | 18.7 | 29.7 | — | — | 16.4 | 1 | — | — | — | — | |
| CTC-NASTArchitecture=NAR2023.05 | — | 27.3 | 31.8 | 38.9 | 27.7 | 32.3 | 33.3 | 26.1 | 18.9 | 29.5 | — | — | 16.2 | 5.67 | — | — | — | — | |
| Fairseq STArchitecture=AR2023.05 | — | 22.7 | 27.2 | 32.9 | 22.7 | 27.3 | 28.1 | 21.9 | 15.3 | 24.8 | — | — | — | — | — | — | — | — | |
| LNA-Dtrainable/total parameters (M)=53.5/76.3, group=Li et al.2021.06 | — | 24.16 | 28.3 | 34.52 | 24.46 | 28.35 | 30.51 | 23.29 | 15.84 | 26.18 | — | — | — | — | — | — | — | — | |
| LNA-Etrainable/total parameters (M)=48.1/76.3, group=Li et al.2021.06 | — | 24.34 | 28.25 | 34.42 | 24.24 | 28.46 | 30.53 | 23.32 | 15.89 | 26.18 | — | — | — | — | — | — | — | — | |
| LNA-E,Dtrainable/total parameters (M)=25.3/76.3, group=Li et al.2021.06 | — | 24.27 | 28.4 | 34.61 | 24.44 | 28.25 | 30.53 | 23.27 | 15.92 | 26.21 | — | — | — | — | — | — | — | — | |
| M³STArchitecture=AR2023.05 | — | 26.4 | 31 | 37.2 | 26.6 | 30.9 | 32.8 | 25.4 | 18.3 | 28.6 | — | — | — | — | — | — | — | — | |
| mBART-FTLearning Strategy=Supervised learning + pre-training2022.10 | — | — | — | — | — | — | — | — | — | — | 38.7 | 23.1 | — | — | — | — | — | — | |
| mBART-OBTLearning Strategy=Unsupervised learning2022.10 | — | — | — | — | — | — | — | — | — | — | 38.5 | 22.2 | — | — | — | — | — | — | |
| NeurSTArchitecture=AR2023.05 | — | 22.8 | 27.4 | 33.3 | 22.9 | 27.2 | 28.7 | 22.2 | 15.1 | 24.9 | — | — | — | — | — | — | — | — | |
| OrthrosArchitecture=NAR, Reference=Inaguma et al., 2021b2023.05 | — | 23.9 | — | 33.1 | — | — | — | — | — | — | — | — | — | 2.39 | — | — | — | — | |
| Orthros-CMLMArchitecture=NAR, Reference=Inaguma et al., 2021a2023.05 | — | 24.1 | 29.2 | 35.1 | 24.4 | 28.6 | — | — | — | — | — | — | — | 2.73 | — | — | — | — | |
| Orthros-CTCArchitecture=NAR, Reference=Inaguma et al., 2021a2023.05 | — | 25.3 | 30.4 | 36.2 | 25.4 | 29.9 | — | — | — | — | — | — | — | 1.14 | — | — | — | — | |
| SeamlessM4T-V2Architecture Category=End-to-End, Zero-shot=true2026.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 18.1 | |
| STEMMArchitecture=AR2023.05 | — | 25.6 | 30.3 | 36.1 | 25.6 | 30.1 | 31 | 24.3 | 17.1 | 27.5 | — | — | — | — | — | — | — | — | |
| TransformerLearning Strategy=Supervised learning2022.10 | — | — | — | — | — | — | — | — | — | — | 33.8 | 15.8 | — | — | — | — | — | — | |
| TransformerArchitecture=MT2023.05 | — | 30.8 | 35.6 | 43.3 | 31.6 | 35.8 | 37.9 | 30.1 | 20 | 31.1 | — | — | 16.5 | — | — | — | — | — | |
| TransformerArchitecture=AR, Reference=Inaguma et al., 2021b, Seq-KD=false2023.05 | — | 23.1 | — | 33.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| TransformerArchitecture=AR, Reference=Inaguma et al., 2021b, Seq-KD=true2023.05 | — | 24.4 | — | 34.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| TransformerArchitecture=AR, Reference=Inaguma et al., 2021a, Seq-KD=false2023.05 | — | 22.8 | 27.8 | 33.3 | 23.3 | 27.3 | — | — | — | — | — | — | — | — | — | — | — | — | |
| TransformerArchitecture=AR, Reference=Inaguma et al., 2021a, Seq-KD=true2023.05 | — | 24.3 | 28.9 | 34.5 | 24.2 | 28.4 | — | — | — | — | — | — | — | — | — | — | — | — | |
| Whisper+NLLB3.3bArchitecture Category=Cascaded, Zero-shot=true2026.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 13.5 | |
| Whisper+Qwen7B-chatArchitecture Category=Cascaded, Zero-shot=true2026.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 22.8 | — | — | |
| XSTNetArchitecture=AR2023.05 | — | 25.5 | 29.6 | 36 | 25.5 | 30 | 31.3 | 25.1 | 16.9 | 27.5 | — | — | — | — | — | — | — | — |