List your company
ResearchFormerly WizWand

Lip Reading on LRW 1.0 (test)

94.8Top-1 AccuracyTranscription Alignment
59.75268.85177.9587.049Sep 3, 2022Dec 21, 2022Apr 9, 2023Jul 27, 2023Nov 13, 2023Mar 1, 2024Jun 18, 2024
Updated 8mo ago

Evaluation results

MethodLinks
Transcription AlignmentTemporal Model=Transformer, Video Hours=157, Word Boundary (WB)=true2024.0694.8
DC-TCN (Ensemble)Word Boundary=true, Pre-training=LRS2&3+AVS, Ensemble=true2022.0994.1
DC-TCN (Ensemble)Word Boundary=true, Pre-training=LiRA(LRS3), Ensemble=true2022.0993.6
DC-TCN (Ensemble)Word Boundary=true, Pre-training=Scratch, Ensemble=true2022.0993.4
Transcription AlignmentTemporal Model=Transformer, Video Hours=157, Word Boundary (WB)=false2024.0693.1
DC-TCNWord Boundary=true, Self-Distillation=false, Pre-training=LRS2&3+AVS2022.0992.9
Training Strategy + CM-AuxTemporal Model=DC-TCN, Video Hours=1459, Word Boundary (WB)=true2024.0692.9
DC-TCNWord Boundary=true, Self-Distillation=false, Pre-training=LiRA(LRS3)2022.0992.3
Training Strategy + LIRATemporal Model=DC-TCN, Video Hours=595, Word Boundary (WB)=true2024.0692.3
DC-TCN (Ensemble)Word Boundary=false, Pre-training=LRS2&3+AVS, Ensemble=true2022.0992.1
DC-TCNWord Boundary=true, Self-Distillation=false, Pre-training=Scratch2022.0992.1
Training StrategyTemporal Model=DC-TCN, Video Hours=157, Word Boundary (WB)=true2024.0692.1
MTLAMTemporal Model=DC-TCN, Video Hours=157, Word Boundary (WB)=false2024.0691.7
DC-TCN (Ensemble)Word Boundary=false, Pre-training=Scratch, Ensemble=true2022.0991.6
DC-TCNWord Boundary=false, Self-Distillation=false, Pre-training=LRS2&3+AVS2022.0991.1
DC-TCNWord Boundary=false, Self-Distillation=false, Pre-training=Scratch2022.0990.4
Training StrategyTemporal Model=DC-TCN, Video Hours=157, Word Boundary (WB)=false2024.0690.4
Koumparoulis et al.Temporal Model=Transformer, Video Hours=157, Word Boundary (WB)=false2024.0689.5
NetVLADTemporal Model=MS-TCN, Video Hours=157, Word Boundary (WB)=false2024.0689.4
ResNet-18 + BLSTMWord Boundary=true2022.0988.8
ResNet-18 + MS-TCN + S.D.Word Boundary=false, Self-Distillation=true2022.0988.5
MVMTemporal Model=MS-TCN, Video Hours=157, Word Boundary (WB)=false2024.0688.5
ResNet-18 + DC-TCNWord Boundary=false2022.0988.4
ResNet-18 + BGRUWord Boundary=true2022.0988.4
Ma et al.Temporal Model=DC-TCN, Video Hours=157, Word Boundary (WB)=false2024.0688.4
Feng et al.Temporal Model=BiGRU, Video Hours=157, Word Boundary (WB)=true2024.0688.4
WPCL + APFFTemporal Model=MS-TCN, Video Hours=157, Word Boundary (WB)=false2024.0688.3
LIRATemporal Model=Conformer, Video Hours=590, Word Boundary (WB)=false2024.0688.1
Born-AgainTemporal Model=MS-TCN, Video Hours=157, Word Boundary (WB)=false2024.0687.9
Feng et al.Temporal Model=BiGRU, Video Hours=157, Word Boundary (WB)=false2024.0686.2
ResNet-18 + MS-TCNWord Boundary=false2022.0985.3
ResNet-18 + BGRU + CutoutWord Boundary=false2022.0985
ResNet-18 + BGRUWord Boundary=false2022.0985
ResNet-18 + BLSTMWord Boundary=false2022.0984.3
2*3D-CNN + BLSTMWord Boundary=false2022.0984.1
ResNet-34 + BLSTMWord Boundary=false2022.0983
3D-CNNWord Boundary=false2022.0961.1