Audio-Visual Automatic Speech Recognition on VisSpeech zero-shot
16.6WERAVFormer
Evaluation results
| Method | Links | |
|---|---|---|
| AVFormerModality=A+V, LibriSpeech PT=true, HowTo100M Pretrained Params=VP + Adapters, HowTo100M PT Data %=52023.03 | 16.6 | |
| BEST-RQModality=A, LibriSpeech PT=true, HowTo100M Pretrained Params=All, HowTo100M PT Data %=1002023.03 | 16.69 | |
| BEST-RQModality=A, LibriSpeech PT=false, HowTo100M Pretrained Params=None2023.03 | 28.62 | |
| AVATARModality=A+V, LibriSpeech PT=true, HowTo100M Pretrained Params=All, HowTo100M PT Data %=1002023.03 | 35.59 | |
| AVATARModality=A+V, LibriSpeech PT=false, HowTo100M Pretrained Params=All, HowTo100M PT Data %=1002023.03 | 35.66 | |
| AVATARModality=A, LibriSpeech PT=true, HowTo100M Pretrained Params=None2023.03 | 65.33 |