Audio-Visual Speech-to-Speech Translation on LRS3-T Fr-En
97.04BLEUVideo
Evaluation results
| Method | Links | ||
|---|---|---|---|
| VideoTextual Dataset (SRC)=None, Textual Dataset (TGT)=None2023.12 | 97.04 | — | |
| Audio+Wav2LipTextual Dataset (SRC)=None, Textual Dataset (TGT)=None2023.12 | 87.73 | — | |
| Unit+Unit2LipTextual Dataset (SRC)=None, Textual Dataset (TGT)=None2023.12 | 83.89 | 4.32 | |
| Unit+U2S+Wav2LipTextual Dataset (SRC)=None, Textual Dataset (TGT)=None2023.12 | 80.71 | 4.18 | |
| ASR+NMT+TTS+Wav2LipTextual Dataset (SRC)=ASR+NMT, Textual Dataset (TGT)=NMT+TTS2023.12 | 52.34 | 4.12 | |
| TransFaceTextual Dataset (SRC)=None, Textual Dataset (TGT)=None2023.12 | 47.55 | 3.88 | |
| TransFace+boundedTextual Dataset (SRC)=None, Textual Dataset (TGT)=None, length_regulation=bounded duration predictor2023.12 | 46.78 | 4.21 | |
| S2ST+Wav2LipTextual Dataset (SRC)=None, Textual Dataset (TGT)=None2023.12 | 45.17 | 3.76 | |
| ST+TTS+Wav2LipTextual Dataset (SRC)=ST, Textual Dataset (TGT)=TTS2023.12 | 42.6 | 3.68 | |
| Translatotron2+Wav2LipTextual Dataset (SRC)=None, Textual Dataset (TGT)=None2023.12 | 36.44 | 3.22 |