Audio-Visual Speech Recognition on MuAViC (test)
89.4Accuracy (Ara)AV-HUBERT
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| AV-HUBERTModality=AVSR, AV Training Hours (Unlabeled)=1,759*, AV Training Hours (Labeled)=1,200, Support #Langs=9, Protocol=Non-Zero Shot2025.03 | 89.4 | 52 | 46.2 | 17.4 | 20.3 | 20.8 | 22.1 | 44.7 | 1.7 | 35 | |
| u-HuBERTModality=AVSR, AV Training Hours (Unlabeled)=1,759*(452*), AV Training Hours (Labeled)=1,200, Support #Langs=9, Protocol=Non-Zero Shot2025.03 | 89.3 | 52.1 | 46.4 | 17.3 | 20.5 | 21.2 | 21.9 | 44.4 | 1.9 | 35 | |
| MMS Zero-shotModality=ASR, AV Training Hours (Unlabeled)=(436K), AV Training Hours (Labeled)=(40K), Support #Langs=1,078+, Protocol=Zero-Shot2025.03 | 84.9 | 31.5 | 47.9 | 17.7 | 33.6 | 19 | 35.5 | 42.8 | 35.7 | 38.9 | |
| Cascaded Zero-AVSRModality=AVSR, AV Training Hours (Unlabeled)=1,759*, AV Training Hours (Labeled)=2,916, Support #Langs=82+, Protocol=Zero-Shot2025.03 | 82.1 | 29.3 | 47.2 | 16.3 | 28.9 | 21.6 | 20.2 | 42.9 | 2.9 | 30.2 | |
| XLAVS-R 300MModality=AVSR, AV Training Hours (Unlabeled)=1,200(436K), AV Training Hours (Labeled)=1,200, Support #Langs=9, Protocol=Non-Zero Shot2025.03 | 81.7 | 44.7 | 24.3 | 10.9 | 14.4 | 12.8 | 13.2 | 32.7 | 2.4 | 26.3 | |
| Zero-AVSRModality=AVSR, AV Training Hours (Unlabeled)=1,759*, AV Training Hours (Labeled)=2,916, Support #Langs=82+, Protocol=Zero-Shot2025.03 | 81.4 | 27.8 | 38.4 | 13.1 | 14.3 | 15.9 | 15.4 | 32.6 | 1.5 | 25.2 | |
| XLAVS-R 2BModality=AVSR, AV Training Hours (Unlabeled)=1,200(436K), AV Training Hours (Labeled)=1,200, Support #Langs=9, Protocol=Non-Zero Shot2025.03 | 79.3 | 44.4 | 19 | 9.1 | 12.3 | 10.6 | 11.2 | 25 | 1.7 | 23.6 |