Audio-Visual Speech Recognition on LRS3 Noisy w.r.t Missing Modality 0.1 Masking Ratio (test)
4.07Word Error Rate (%)Auto-AVSR
Evaluation Results
| Method | Links | |
|---|---|---|
| Auto-AVSRModality=A+V, Labeled Training Data (hrs)=3,4482026.06 | 4.07 | |
| MMS-LlamaModality=A+V, Labeled Training Data (hrs)=1,7592026.06 | 4.53 | |
| M2S-AVSRModality=A+V, Labeled Training Data (hrs)=ms1,759, Multi-view Synthesis=true2026.06 | 4.6 | |
| M2S-AVSRModality=A+V, Labeled Training Data (hrs)=1,7592026.06 | 4.64 | |
| Llama-AVSRModality=A+V, Labeled Training Data (hrs)=1,7592026.06 | 4.73 | |
| MMS-LlamaModality=A+V, Labeled Training Data (hrs)=4332026.06 | 4.79 | |
| M2S-AVSRModality=A+V, Labeled Training Data (hrs)=ms433, Multi-view Synthesis=true2026.06 | 5.3 | |
| M2S-AVSRModality=A+V, Labeled Training Data (hrs)=4332026.06 | 5.38 | |
| Whisper-FlamingoModality=A+V, Labeled Training Data (hrs)=ms1,759, Multi-view Synthesis=true2026.06 | 6.16 | |
| Whisper-FlamingoModality=A+V, Labeled Training Data (hrs)=1,7592026.06 | 6.17 | |
| CMAModality=A+V, Labeled Training Data (hrs)=4332026.06 | 6.33 | |
| Whisper-FlamingoModality=A+V, Labeled Training Data (hrs)=ms433, Multi-view Synthesis=true2026.06 | 6.37 | |
| Whisper-FlamingoModality=A+V, Labeled Training Data (hrs)=4332026.06 | 6.72 | |
| AV-HuBERTModality=A+V, Labeled Training Data (hrs)=4332026.06 | 7.29 | |
| AV-RelScoreModality=A+V, Labeled Training Data (hrs)=4332026.06 | 8.81 | |
| V-CAFEModality=A+V, Labeled Training Data (hrs)=4332026.06 | 13.27 |