Speech Translation on FLEURS X→En (test)
37.4BLEU (Average)Gemin-3.1-Pro
Evaluation results
| Method | Links | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Gemin-3.1-Provariant=Pro2026.04 | 37.4 | 42.1 | 27.7 | 43.9 | 35.3 | 30.3 | 38 | 41.6 | 39.3 | 30.8 | 29.5 | 47.7 | — | 41.7 | 36.1 | 30.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 34.9 | — | — | 42.7 | 44.8 | 45.1 | 32 | 36.7 | 34.2 | 39.5 | 31.2 | 32.9 | 42.6 | 46.6 | 44.7 | 42.5 | 36.7 | 30.4 | 35.1 | 40.1 | 45 | 37.2 | 20.6 | 41.3 | 34.9 | 46.2 | 38.9 | 40.1 | 39.6 | 40.3 | 38 | 35.3 | 38.4 | 35.7 | 33.1 | 24.8 | 38 | 43.1 | 36.7 | 38.3 | 42.3 | 39.9 | 34.7 | 42.6 | 34.5 | 23.7 | |
| Qwen3.5-Omni-Plusvariant=Plus2026.04 | 37 | 42.9 | 30.3 | 44.6 | 36 | 32.3 | 40.2 | 43.5 | 38.8 | 34.4 | 30.4 | 49.4 | — | 42 | 36.4 | 32.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 31.2 | — | — | 43.1 | 42.9 | 45.7 | 34.1 | 38.6 | 34.1 | 40.3 | 33.3 | 35.5 | 43.5 | 47.2 | 45.4 | 39.7 | 36.9 | 32.1 | 31.5 | 42.1 | 42.7 | 37 | 23.1 | 39.6 | 32 | 46.6 | 37.3 | 35.7 | 40.8 | 33.4 | 39.5 | 35.5 | 35.9 | 34.4 | 26.4 | 22.2 | 33.7 | 43.3 | 33.5 | 33 | 43.5 | 39.7 | 31.7 | 35 | 33.9 | 25 | |
| Gemini-3-Flashvariant=Flash2026.04 | 36.6 | 42.3 | 26.3 | 43.9 | 35.4 | 30.4 | 38.1 | 41.3 | 39.2 | 31 | 29.4 | 47.5 | — | 41.4 | 35.4 | 29.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 34.2 | — | — | 42.4 | 42.4 | 44.9 | 32.1 | 36.5 | 33 | 38.9 | 30.4 | 32.6 | 41.4 | 44.7 | 42.7 | 39.9 | 35.5 | 29.7 | 35.8 | 39.5 | 44.3 | 35.5 | 19.9 | 40 | 34.3 | 45.5 | 38.2 | 38.3 | 38.6 | 39.7 | 36.9 | 33.3 | 36.5 | 35.5 | 33.6 | 23.7 | 37.9 | 41.9 | 35.5 | 36.5 | 41.5 | 38.8 | 33.2 | 40.5 | 33.3 | 23.3 | |
| GenTranslate2024.02 | 34 | 39.9 | 39.4 | 41.6 | 32.8 | 31.2 | 35.9 | 40.6 | 34.9 | 32.1 | 22.8 | 45 | 24.1 | 36.9 | 27.4 | 25.7 | 63.1 | 61.2 | 64.9 | 57 | 57.1 | 59.7 | 64 | 59.1 | 58 | 47.6 | 67.2 | 49.7 | 60.8 | 51.6 | 52 | 58.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Qwen3.5-Omni-Flashvariant=Flash2026.04 | 33.5 | 40 | 29.9 | 44.1 | 32.5 | 31.3 | 36.8 | 42 | 36.2 | 31.9 | 29.2 | 48.2 | — | 40.1 | 34.9 | 31.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 25.7 | — | — | 40.8 | 41.1 | 43.1 | 32.4 | 37.2 | 32.4 | 39.1 | 32.2 | 31.5 | 42.1 | 45.2 | 44 | 36.4 | 35 | 30.5 | 27.5 | 39.3 | 40.9 | 35.1 | 19.9 | 36 | 27.6 | 42.3 | 26.3 | 28.3 | 38.6 | 28.3 | 36.3 | 29.6 | 28.4 | 26.9 | 19.8 | 17.1 | 25.2 | 39.6 | 25.7 | 26.9 | 39.7 | 38.4 | 26.5 | 27.4 | 29 | 22 | |
| SeamlessM4TMode=ASR+MT2024.02 | 31.9 | 38.9 | 37 | 39.7 | 29 | 27.7 | 34.1 | 37.7 | 33.9 | 28.9 | 21.7 | 42.3 | 23.7 | 34 | 24.9 | 24.4 | 62.7 | 60 | 63.8 | 55 | 56 | 58.7 | 62.4 | 58.8 | 57 | 47.9 | 65.9 | 49.8 | 59.2 | 50.5 | 51.5 | 57.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| NLLBType=MT, Model Size=Large2024.02 | 24.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| NLLBType=MT, Model Size=Medium2024.02 | 22.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SEAMLESSM4TType=E2E-ST, Model Size=Large2024.02 | 21.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ZEROSWOTType=ZS-ST, Model Size=Large, Training Data=Common Voice + MUST-C(ASR) + LibriSpeech2024.02 | 20.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AED-ASR + NLLBType=Cascade-ST, Model Size=Large2024.02 | 20.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ZEROSWOTType=ZS-ST, Model Size=Large2024.02 | 20.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SEAMLESSM4TType=E2E-ST, Model Size=Medium2024.02 | 19.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ZEROSWOTType=ZS-ST, Model Size=Medium, Training Data=Common Voice + MUST-C(ASR) + LibriSpeech2024.02 | 18.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ZEROSWOTType=ZS-ST, Model Size=Medium2024.02 | 18.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| AED-ASR + NLLBType=Cascade-ST, Model Size=Medium2024.02 | 17.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Att(sem)2026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 62.59 | 60.88 | 55.4 | 60.39 | 14.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Att(sem+spk)2026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 61.55 | 59.59 | 55.17 | 60.28 | 16.38 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Baseline2024.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 21.9 | 33.7 | 23.9 | 16.6 | 22.4 | 16.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Llama 3Model size=8B2024.07 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 29.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Llama 3Model size=70B2024.07 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 33.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Real2024.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 24.5 | 36.5 | 26.4 | 19 | 24.4 | 17.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SeamlessM4TVersion=v22024.07 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 28.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Self-train.2024.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 22.1 | 34 | 24.1 | 16.3 | 22.5 | 16.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SONAR2026.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 22.36 | 39.98 | 54.12 | 27.27 | 8.19 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| STAR2024.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 23.3 | 35.9 | 24.8 | 17.6 | 23.4 | 17.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| WhisperVersion=v22024.07 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 21.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |