Lip Reading on LRS3 (test)
17WERViT-3D
Evaluation Results
| Method | Links | |
|---|---|---|
| ViT-3DPhoneme-based=false, Input=Video, Extra Data=true, Total Hours=900002025.07 | 17 | |
| V-ASRPhoneme-based=true, Input=Video, LLM=NLLB(1.3B), Extra Data=false, Total Hours=4382025.07 | 17.3 | |
| VALLRPhoneme-based=true, Input=Video, LLM=LLaMA, Extra Data=false, Total Hours=302025.07 | 18.7 | |
| Auto-AVSRPhoneme-based=false, Input=Video, Extra Data=true, Total Hours=34482025.07 | 19.1 | |
| PV-ASRPhoneme-based=true, Input=Video+117-Points, LLM=NLLB(1.3B), Extra Data=false, Total Hours=4382025.07 | 20.3 | |
| AV-ViTMode=Full-Shot, Labeled Utts Video(hrs)=90000, Publicly Available Dataset=false2023.06 | 25.9 | |
| OpenSRMode=Full-Shot, Unlabeled Utts=LRS3+VC2-EN, Labeled Utts Video(hrs)=4332023.06 | 28.5 | |
| AV-HubertMode=Full-Shot, Unlabeled Utts=LRS3+VC2-EN, Labeled Utts Video(hrs)=4332023.06 | 28.6 | |
| OpenSRMode=Zero-Shot, Unlabeled Utts=LRS3+VC2-EN2023.06 | 30.6 | |
| SubMode=Full-Shot, Labeled Utts Video(hrs)=2676, Publicly Available Dataset=false2023.06 | 30.7 | |
| VTPPhoneme-based=false, Input=Video, Extra Data=true, Total Hours=26762025.07 | 30.7 | |
| VisualSRMode=Full-Shot, Labeled Utts Video(hrs)=14592023.06 | 31.5 | |
| Auto-AVSRPhoneme-based=false, Input=Video, Extra Data=true, Total Hours=8182025.07 | 33 | |
| RecurrentNNMode=Full-Shot, Labeled Utts Video(hrs)=31000, Publicly Available Dataset=false2023.06 | 33.6 | |
| CM-auxPhoneme-based=false, Input=Video, Extra Data=false, Total Hours=4382025.07 | 36.3 | |
| VTPPhoneme-based=false, Input=Video, Extra Data=true, Total Hours=6982025.07 | 40.6 | |
| LiraMode=Full-Shot, Labeled Utts Video(hrs)=5902023.06 | 43.3 | |
| Hyb.-Conf.Phoneme-based=false, Input=Video, Extra Data=true, Total Hours=5902025.07 | 43.3 | |
| ASSTGCNPhoneme-based=false, Input=Video+38-Points, Extra Data=false, Total Hours=4382025.07 | 62.7 | |
| P-ASRPhoneme-based=true, Input=117-Points, LLM=NLLB(1.3B), Extra Data=false, Total Hours=4382025.07 | 66.4 |