Audio-Visual Speech Recognition on LRS3 Noisy w.r.t Missing Modality 0.3 Masking Ratio (test)
5.77WER (%)M2S-AVSR
Evaluation Results
| Method | Links | |
|---|---|---|
| M2S-AVSRModality=A+V, Labeled Training Data (hrs)=ms1,759, Multi-view Synthesis=true2026.06 | 5.77 | |
| M2S-AVSRModality=A+V, Labeled Training Data (hrs)=1,7592026.06 | 5.84 | |
| MMS-LlamaModality=A+V, Labeled Training Data (hrs)=1,7592026.06 | 5.9 | |
| M2S-AVSRModality=A+V, Labeled Training Data (hrs)=ms433, Multi-view Synthesis=true2026.06 | 6.01 | |
| Auto-AVSRModality=A+V, Labeled Training Data (hrs)=3,4482026.06 | 6.11 | |
| MMS-LlamaModality=A+V, Labeled Training Data (hrs)=4332026.06 | 6.17 | |
| Llama-AVSRModality=A+V, Labeled Training Data (hrs)=1,7592026.06 | 6.75 | |
| M2S-AVSRModality=A+V, Labeled Training Data (hrs)=4332026.06 | 7.23 | |
| Whisper-FlamingoModality=A+V, Labeled Training Data (hrs)=1,7592026.06 | 8.22 | |
| Whisper-FlamingoModality=A+V, Labeled Training Data (hrs)=ms1,759, Multi-view Synthesis=true2026.06 | 8.42 | |
| Whisper-FlamingoModality=A+V, Labeled Training Data (hrs)=4332026.06 | 8.52 | |
| Whisper-FlamingoModality=A+V, Labeled Training Data (hrs)=ms433, Multi-view Synthesis=true2026.06 | 8.55 | |
| CMAModality=A+V, Labeled Training Data (hrs)=4332026.06 | 9.35 | |
| AV-RelScoreModality=A+V, Labeled Training Data (hrs)=4332026.06 | 11.13 | |
| AV-HuBERTModality=A+V, Labeled Training Data (hrs)=4332026.06 | 11.14 | |
| V-CAFEModality=A+V, Labeled Training Data (hrs)=4332026.06 | 14.87 |