Speech Translation on FLEURS (test)
0.853COMET (Low)Speech-charSONAR-75 - DUAL
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Speech-charSONAR-75 - DUALMethod Category=Zero-shot E2E ST (ours), Text Encoder Tokenization=chars, Total Params=2.3B, Adapter Train Params=2.5M, Adapter Train Data=CV+FLEURS2025.05 | 0.853 | 0.906 | 0.91 | 0.889 | |
| Speech-charSONAR-75 - PRETRMethod Category=Zero-shot E2E ST (ours), Text Encoder Tokenization=chars, Total Params=2.3B, Adapter Train Params=0.2M, Adapter Train Data=CV+FLEURS2025.05 | 0.852 | 0.9 | 0.901 | 0.884 | |
| Speech-charSONAR-75 - PRETRMethod Category=Zero-shot E2E ST (ours), Text Encoder Tokenization=chars, Total Params=2.3B, Adapter Train Params=0.2M, Adapter Train Data=CV2025.05 | 0.837 | 0.893 | 0.894 | 0.875 | |
| MMS + charSONAR-75Method Category=Cascade ST (ours), Text Encoder Tokenization=chars, Total Params=2.3B2025.05 | 0.833 | 0.889 | 0.875 | 0.866 | |
| SEAMLESSM4T-LARGE-v2Method Category=Supervised E2E ST (previous), Total Params=2.3B2025.05 | 0.829 | 0.889 | 0.901 | 0.873 | |
| MMS + SONAR-75Method Category=Cascade ST (ours), Text Encoder Tokenization=subwords, Total Params=2.3B2025.05 | 0.811 | 0.87 | 0.854 | 0.845 | |
| MMS + NLLB-200Method Category=Cascade ST (previous), Text Encoder Tokenization=subwords, Total Params=2.3B2025.05 | 0.786 | 0.834 | 0.822 | 0.814 | |
| Speech-charSONAR-75 - PRETRMethod Category=Zero-shot E2E ST (ours), Text Encoder Tokenization=chars, Total Params=2.3B, Adapter Train Params=02025.05 | 0.772 | 0.833 | 0.831 | 0.812 | |
| MMS + SONAR-200Method Category=Cascade ST (previous), Text Encoder Tokenization=subwords, Total Params=2.3B2025.05 | 0.757 | 0.839 | 0.824 | 0.807 | |
| WHISPER + charSONAR-75Method Category=Cascade ST (ours), Text Encoder Tokenization=chars, Total Params=2.8B2025.05 | 0.755 | 0.893 | 0.882 | 0.843 | |
| WHISPER + SONAR-75Method Category=Cascade ST (ours), Text Encoder Tokenization=subwords, Total Params=2.8B2025.05 | 0.721 | 0.871 | 0.865 | 0.819 | |
| WHISPER + NLLB-200Method Category=Cascade ST (previous), Text Encoder Tokenization=subwords, Total Params=2.8B2025.05 | 0.717 | 0.87 | 0.863 | 0.817 | |
| WHISPER + SONAR-200Method Category=Cascade ST (previous), Text Encoder Tokenization=subwords, Total Params=2.8B2025.05 | 0.684 | 0.869 | 0.861 | 0.804 | |
| Speech-charSONAR-75 - DUALMethod Category=Zero-shot E2E ST (ours), Text Encoder Tokenization=chars, Total Params=2.3B, Adapter Train Params=2.5M, Adapter Train Data=CV2025.05 | 0.615 | 0.899 | 0.902 | 0.805 | |
| WHISPER-LARGE-V3Method Category=Supervised E2E ST (previous), Total Params=1.5B2025.05 | 0.598 | 0.754 | 0.79 | 0.714 |