Automatic Speech Recognition on LRS3 Clean original (test)
0.68WERWhisper-Large, Fine-tuned w/o noise
Evaluation Results
| Method | Links | |
|---|---|---|
| Whisper-Large, Fine-tuned w/o noiseBeam search=1, Labeled AV Training Hrs=1,7592024.06 | 0.68 | |
| Fast ConformerNoise Dataset=NoiseX, Unlabeled AV Training Hrs=3,1162024.06 | 0.7 | |
| Llama-AVSRNoise Dataset=NoiseX, Unlabeled AV Training Hrs=1,759, Labeled AV Training Hrs=1,7592024.06 | 0.81 | |
| Whisper-Large, Fine-tuned w/ noiseBeam search=1, Labeled AV Training Hrs=1,7592024.06 | 0.85 | |
| LP ConformerNoise Dataset=NoiseX, Labeled AV Training Hrs=100k2024.06 | 0.9 | |
| AutoAVSRNoise Dataset=NoiseX, Unlabeled AV Training Hrs=1,9022024.06 | 1 | |
| AutoAVSRNoise Dataset=NoiseX, Unlabeled AV Training Hrs=3,4482024.06 | 1 | |
| Whisper-Large, Fine-tuned w/o noiseBeam search=1, Labeled AV Training Hrs=4332024.06 | 1 | |
| Llama-AVSRNoise Dataset=NoiseX, Unlabeled AV Training Hrs=1,759, Labeled AV Training Hrs=4332024.06 | 1.1 | |
| Whisper-Large, Fine-tuned w/ noiseBeam search=1, Labeled AV Training Hrs=4332024.06 | 1.1 | |
| Whisper-Large, Fine-tuned w/o noiseBeam search=15, Labeled AV Training Hrs=4332024.06 | 1.3 | |
| u-HuBERTNoise Dataset=LRS3+, Unlabeled AV Training Hrs=1,759, Labeled AV Training Hrs=4332024.06 | 1.4 | |
| Whisper-Large, Fine-tuned w/o noiseBeam search=15, Labeled AV Training Hrs=1,7592024.06 | 1.5 | |
| VIT 3DNoise Dataset=NoiseX, Labeled AV Training Hrs=90k2024.06 | 1.6 | |
| Fast ConformerNoise Dataset=NoiseX, Unlabeled AV Training Hrs=4352024.06 | 1.6 | |
| AV-HUBERTNoise Dataset=LRS3+, Unlabeled AV Training Hrs=1,759, Labeled AV Training Hrs=4332024.06 | 1.6 | |
| Whisper-Large, Fine-tuned w/ noiseBeam search=15, Labeled AV Training Hrs=1,7592024.06 | 2 | |
| Whisper-Large Zero-shotBeam search=152024.06 | 2.1 | |
| Whisper-Large Zero-shotBeam search=12024.06 | 2.3 | |
| Whisper-Large, Fine-tuned w/ noiseBeam search=15, Labeled AV Training Hrs=4332024.06 | 2.3 | |
| AV2vecNoise Dataset=MUSAN, Unlabeled AV Training Hrs=433, Labeled AV Training Hrs=4332024.06 | 2.7 |