Speech-to-speech translation on Multi-domain En->Es (Europarl-ST, MuST-C)
36.9Score (Europarl-ST)S2SpecT2 + t-mBART
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| S2SpecT2 + t-mBARTSystem category=Direct speech-to-spectrogram systems, Decoder layer configuration=12L→6L, Decoder pre-training=t-mBART2022.12 | 36.9 | 34.3 | 35.6 | |
| ASR→MT→TTS (C1')System category=Cascaded systems, Note=Improved by R-Drop and hyperparameter tuning2022.12 | 36.8 | 30.8 | 33.8 | |
| S2TT-TTS (C2')System category=Cascaded systems, Note=Improved by R-Drop and hyperparameter tuning2022.12 | 36.4 | 33.4 | 34.9 | |
| S2SpecT2System category=Direct speech-to-spectrogram systems, Decoder layer configuration=6L→6L2022.12 | 35.6 | 33.5 | 34.6 | |
| UnitY + t-mBARTSystem category=Direct speech-to-unit systems, Decoder layer configuration=12L→2L, Decoder pre-training=t-mBART2022.12 | 35.3 | 34.1 | 34.7 | |
| UnitYSystem category=Direct speech-to-unit systems, Decoder layer configuration=6L→6L2022.12 | 35.1 | 33.7 | 34.4 | |
| S2UT + u-mBART (C5')System category=Direct speech-to-unit systems, Decoder pre-training=u-mBART, Note=Improved by hyperparameter tuning and checkpoint averaging2022.12 | 33.5 | 33.3 | 33.4 | |
| S2UT + u-mBARTSystem category=Direct speech-to-unit systems, Decoder pre-training=u-mBART2022.12 | 32.7 | 32.1 | 32.4 | |
| S2TT-TTSSystem category=Cascaded systems2022.12 | 32.6 | 30.1 | 31.4 | |
| ASR→MT→TTSSystem category=Cascaded systems2022.12 | 28.8 | 34.2 | 31.5 |