English Transcription on LRS2 clean (test)
1.3ASR WERWhisper-Medium FT
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Whisper-Medium FTAV Training Hrs (Labeled)=223, Beam size=1, Mode=Trained with noise2024.06 | 1.3 | — | |
| AutoAVSRAV Training Hrs (Labeled)=34482024.06 | 1.5 | 1.5 | |
| USRAV Training Hrs (Unlabeled)=1759, AV Training Hrs (Labeled)=2232024.06 | 1.9 | 1.9 | |
| Efficient ConformerAV Training Hrs (Labeled)=8182024.06 | 2.4 | 2.3 | |
| RAVEnAV Training Hrs (Unlabeled)=1759, AV Training Hrs (Labeled)=2232024.06 | 2.5 | — | |
| UniAVSRAV Training Hrs (Labeled)=3802024.06 | 2.7 | 2.6 | |
| CM-seq2seqAV Training Hrs (Labeled)=3802024.06 | 3.9 | 3.7 | |
| Whisper-MediumEvaluation Protocol=Zero-shot, Beam size=12024.06 | 5.2 | — | |
| Whisper-LargeEvaluation Protocol=Zero-shot, Beam size=12024.06 | 5.5 | — | |
| TDNNAV Training Hrs (Labeled)=2232024.06 | 6.7 | 5.9 | |
| CTC/AttentionAV Training Hrs (Labeled)=3802024.06 | 8.3 | 7 | |
| TM-seq2seqAV Training Hrs (Labeled)=2232024.06 | 9.7 | 8.5 | |
| Whisper-FlamingoAV Training Hrs (Unlabeled)=1759, AV Training Hrs (Labeled)=223+, Beam size=1, Mode=Trained with noise2024.06 | — | 1.4 |