List your company
ResearchFormerly WizWand

Visual Speech Recognition on LRS3

0.009WERAuto-AVSR
-0.014560.144470.30350.46253Jun 18, 2024Oct 3, 2024Jan 18, 2025May 5, 2025Aug 20, 2025Dec 5, 2025Mar 23, 2026
Updated 7mo ago

Evaluation results

MethodLinks
Auto-AVSRLabelled hours=3,448, Language model=true, Shared params=false2024.110.009
LP ConfLabelled hours=100,000, Language model=false, Shared params=false2024.110.009
Auto-AVSRLabelled hours=1,902, Language model=true, Shared params=false2024.110.01
USRLabelled hours=433, Unlabelled hours=1,326, Language model=true, Shared params=true2024.110.011
ViT3D-CMLabelled hours=90,000, Language model=false, Shared params=false2024.110.016
RNN-TLabelled hours=31,000, Language model=true, Shared params=false2024.110.045
LP ConformerVideo Hours=100,000, LM=false, Model Size=base-size2024.060.128
LP ConfLabelled hours=100,000, Language model=false, Shared params=false2024.110.128
LP ConfLabelled hours=100,000, Lang. model=false, Shared params=false2026.020.128
SynthVSRLabelled hours=6,720, Language model=false, Shared params=false2024.110.169
SynthVSRLabelled hours=6,720, Lang. model=true, Shared params=false2026.020.169
ViT 3DVideo Hours=90,000, LM=false, Model Size=base-size2024.060.17
ViT3D-CMLabelled hours=90,000, Language model=false, Shared params=false2024.110.17
ViT3D-CMLabelled hours=90,000, Lang. model=false, Shared params=false2026.020.17
USR 2.0Labelled hours=656, Unlabelled hours=2,649, Lang. model=false, Shared params=true2026.020.176
AutoAVSRVideo Hours=3,448, LM=true, Model Size=base-size2024.060.191
Auto-AVSRLabelled hours=3,448, Language model=true, Shared params=false2024.110.191
Auto-AVSRLabelled hours=3,448, Lang. model=true, Shared params=false2026.020.191
BRAVEn w/ STLabelled hours=433, Unlabelled hours=2,649, Lang. model=true, Shared params=false2026.020.201
SyncVSRVideo Hours=1,992, LM=true, Model Size=base-size2024.060.215
USRLabelled hours=433, Unlabelled hours=1,326, Language model=true, Shared params=true2024.110.215
USRLabelled hours=433, Unlabelled hours=1,326, Lang. model=true, Shared params=true2026.020.215
RAVEn-LargeVideo Hours=1,992/1,759, LM=true, Model Size=large-size2024.060.231
RAVEn w/ STLabelled hours=433, Unlabelled hours=1,326, Language model=true, Shared params=false2024.110.231
RAVEn w/ STLabelled hours=433, Unlabelled hours=1,326, Lang. model=true, Shared params=false2026.020.231
SyncVSRVideo Hours=1,992, LM=false, Model Size=base-size2024.060.234
Auto-AVSRLabelled hours=1,902, Language model=true, Shared params=false2024.110.235
Auto-AVSRLabelled hours=1,902, Lang. model=true, Shared params=false2026.020.235
Llama-AVSRLabelled hours=680,000, Lang. model=true, Shared params=false, Encoder=Whisper2026.020.24
Llama-AVSRPreprocessing=Lip crop & align2026.030.24
RAVEn-LargeVideo Hours=1,992/1,759, LM=false, Model Size=large-size2024.060.244
VATLM-LargeVideo Hours=1,992/1,759, LM=false, Model Size=large-size2024.060.262
AV-HUBERT-LargeVideo Hours=1,992/1,759, LM=false, Model Size=large-size2024.060.269
SyncVSRVideo Hours=661, LM=true, Model Size=base-size2024.060.281
AV-HUBERT w/ STLabelled hours=433, Unlabelled hours=1,326, Language model=false, Shared params=false2024.110.286
AV-HuBERT w/ STLabelled hours=433, Unlabelled hours=1,326, Lang. model=false, Shared params=false2026.020.286
SyncVSRVideo Hours=661, LM=false, Model Size=base-size2024.060.304
VTPVideo Hours=2,676, LM=true, Model Size=base-size2024.060.307
VTPLabelled hours=2,676, Language model=true, Shared params=false2024.110.307
VTPLabelled hours=2,676, Lang. model=true, Shared params=false2026.020.307
SyncVSRVideo Hours=223/438, LM=true, Model Size=base-size2024.060.312
CM-AuxVideo Hours=1,459, LM=true, Model Size=base-size2024.060.315
AutoAVSRVideo Hours=818, LM=true, Model Size=base-size2024.060.33
AutoAVSRPreprocessing=Lip crop & align2026.030.33
RAVEnVideo Hours=1,992/1,759, LM=false, Model Size=base-size2024.060.331
SyncVSRVideo Hours=223/438, LM=false, Model Size=base-size2024.060.333
HumanOmni-SpeakerPreprocessing=Raw video2026.030.334
RNN-TLabelled hours=31,000, Language model=true, Shared params=false2024.110.336
RNN-TLabelled hours=31,000, Lang. model=false, Shared params=true2026.020.336
VATLMVideo Hours=1,992/1,759, LM=false, Model Size=base-size2024.060.342
CM-AuxVideo Hours=818, LM=true, Model Size=base-size2024.060.347
AV-HUBERTVideo Hours=1,992/1,759, LM=false, Model Size=base-size2024.060.348
AV-HuBERT BasePreprocessing=Lip crop & align2026.030.348
AutoAVSRVideo Hours=438, LM=true, Model Size=base-size2024.060.363
CM-AuxVideo Hours=223/438, LM=true, Model Size=base-size2024.060.379
RAVEnVideo Hours=438, LM=true, Model Size=base-size2024.060.391
VTPVideo Hours=698, LM=true, Model Size=base-size2024.060.406
CM-Seq2SeqVideo Hours=223/438, LM=true, Model Size=base-size2024.060.469
V2PLabelled hours=3,886, Language model=false, Shared params=false2024.110.551
V2PLabelled hours=3,886, Lang. model=false, Shared params=false2026.020.551
TM-Seq2SeqVideo Hours=1,391, LM=true, Model Size=base-size2024.060.589
KD-Seq2SeqVideo Hours=818, LM=false, Model Size=base-size2024.060.59
KD + CTCVideo Hours=995, LM=true, Model Size=base-size2024.060.598