List your company
ResearchFormerly WizWand

Audio-visual Speech Recognition on LRS2 (test)

1.3WERUSR 2.0
0.8044.1527.510.848Jan 6, 2020Jan 13, 2021Jan 21, 2022Jan 29, 2023Feb 6, 2024Feb 13, 2025Feb 22, 2026
Updated 8mo ago

Evaluation results

MethodLinks
USR 2.0Labelled hours=656, Unlabelled hours=2,649, Lang. model=No, Shared params=Yes, Training Protocol=Self/semi-supervised2026.021.3
Whisper-FlamingoTraining Hours=1992, fine-tuning=fine-tuned with AV-HuBERT (video) and Whisper (audio)2026.021.4
Whisper-FlamingoLabelled hours=680,000, Unlabelled hours=-, Lang. model=Yes, Shared params=No, Training Protocol=Supervised2026.021.4
Auto-AVSRLabelled hours=3,448, Language model=true, Shared params=false2024.111.5
Auto-AVSRTraining Hours=34482026.021.5
Auto-AVSRLabelled hours=3,448, Unlabelled hours=-, Lang. model=Yes, Shared params=No, Training Protocol=Supervised2026.021.5
USRLabelled hours=223, Unlabelled hours=1,759, Language model=false, Shared params=true2024.111.9
USRLabelled hours=223, Unlabelled hours=1,759, Language model=true, Shared params=true2024.111.9
USRLabelled hours=223, Unlabelled hours=1,759, Lang. model=Yes, Shared params=Yes, Training Protocol=Self/semi-supervised2026.021.9
Our ModelLanguage Model=None2022.022.6
Uni-AVSRLabelled hours=223, Unlabelled hours=60,000, Language model=false, Shared params=false2024.112.6
Uni-AVSRLabelled hours=223, Unlabelled hours=60,000, Lang. model=No, Shared params=No, Training Protocol=Self/semi-supervised2026.022.6
OpenSRLabeled Video Utt (hrs)=224, Labeled Audio Utt (hrs)=2242023.062.7
OpenSRLabeled Video Utt (hrs)=29, Labeled Audio Utt (hrs)=2242023.062.8
CoBRATraining Hours=6642026.022.8
Shi et al.Labeled Video Utt (hrs)=224, Labeled Audio Utt (hrs)=2242023.063.1
OpenSRLabeled Video Utt (hrs)=0, Labeled Audio Utt (hrs)=224, Evaluation protocol=zero-shot2023.063.3
E2E ConformerLanguage Model=Powerful Transformer2022.023.7
Ma et al.Labeled Video Utt (hrs)=224, Labeled Audio Utt (hrs)=2242023.063.7
CM-seq2seqLabelled hours=380, Language model=true, Shared params=false2024.113.7
CM-seq2seqTraining Hours=381, mode=baseline2026.023.7
CM-seq2seqLabelled hours=380, Unlabelled hours=-, Lang. model=Yes, Shared params=No, Training Protocol=Supervised2026.023.7
LF-MMI TDNNLanguage Model=External2022.025.9
Yu et al.Labeled Video Utt (hrs)=224, Labeled Audio Utt (hrs)=2242023.065.9
LF-MMI TDNNModality=Audio-Visual2020.015.93
CTC/AttentionModality=Audio-Visual2020.017
Petridis et al.Labeled Video Utt (hrs)=381, Labeled Audio Utt (hrs)=3812023.067
TM-CTCLanguage Model=External2022.028.2
TM-seq2seqLanguage Model=External2022.028.5
Afouras et al.Labeled Video Utt (hrs)=1428, Labeled Audio Utt (hrs)=14282023.068.5
TM-DCM2022.028.6
CE TDNNModality=Audio-Visual2020.018.95
TM-Seq2seqModality=Audio-Visual2020.019.4
TM-CTCModality=Audio-Visual2020.0113.7