Visual Speech Recognition on LRS3 Low-Resource 30h labelled v1 (test)
0.024WERUSR
Evaluation Results
| Method | Links | |
|---|---|---|
| USRPre-train data=LRS3+Vox2, Shared params=true, Model scale=Large2024.11 | 0.024 | |
| USRPre-train data=LRS3+Vox2, Shared params=true, Model scale=Base(+)2024.11 | 0.025 | |
| VATLMPre-train data=LRS3+Vox2, Shared params=false, Model scale=Large2024.11 | 0.027 | |
| AV-data2vecPre-train data=LRS3+Vox2, Shared params=false, Model scale=Large2024.11 | 0.027 | |
| USRPre-train data=LRS3, Shared params=true, Model scale=Base(+)2024.11 | 0.03 | |
| AV-data2vecPre-train data=LRS3+Vox2, Shared params=false, Model scale=Base(+)2024.11 | 0.033 | |
| AV-HuBERTPre-train data=LRS3+Vox2, Shared params=false, Model scale=Large2024.11 | 0.033 | |
| VATLMPre-train data=LRS3+Vox2, Shared params=false, Model scale=Base(+)2024.11 | 0.034 | |
| VATLMPre-train data=LRS3, Shared params=false, Model scale=Base(+)2024.11 | 0.036 | |
| BRAVEnPre-train data=LRS3, Shared params=false, Model scale=Base(+)2024.11 | 0.04 | |
| AV-HuBERTPre-train data=LRS3+Vox2, Shared params=false, Model scale=Base(+)2024.11 | 0.04 | |
| AV-data2vecPre-train data=LRS3, Shared params=false, Model scale=Base(+)2024.11 | 0.042 | |
| AV-HuBERTPre-train data=LRS3, Shared params=false, Model scale=Base(+)2024.11 | 0.047 | |
| USRPre-train data=LRS3+Vox2, Shared params=true, Model scale=Large2024.11 | 0.269 | |
| USRPre-train data=LRS3+Vox2, Shared params=true, Model scale=Base(+)2024.11 | 0.284 | |
| AV-data2vecPre-train data=LRS3+Vox2, Shared params=false, Model scale=Large2024.11 | 0.308 | |
| BRAVEnPre-train data=LRS3+Vox2, Shared params=false, Model scale=Large2024.11 | 0.308 | |
| Lip2VecPre-train data=LRS3+Vox2, Shared params=false, Model scale=Large2024.11 | 0.312 | |
| VATLMPre-train data=LRS3+Vox2, Shared params=false, Model scale=Large2024.11 | 0.316 | |
| AV-HuBERTPre-train data=LRS3+Vox2, Shared params=false, Model scale=Large2024.11 | 0.325 | |
| RAVEnPre-train data=LRS3+Vox2, Shared params=false, Model scale=Large2024.11 | 0.325 | |
| BRAVEnPre-train data=LRS3+Vox2, Shared params=false, Model scale=Base(+)2024.11 | 0.351 | |
| USRPre-train data=LRS3, Shared params=true, Model scale=Base(+)2024.11 | 0.36 | |
| AV-data2vecPre-train data=LRS3+Vox2, Shared params=false, Model scale=Base(+)2024.11 | 0.378 | |
| RAVEnPre-train data=LRS3+Vox2, Shared params=false, Model scale=Base(+)2024.11 | 0.402 | |
| Lip2VecPre-train data=LRS3+Vox2, Shared params=false, Model scale=Base(+)2024.11 | 0.406 | |
| VATLMPre-train data=LRS3+Vox2, Shared params=false, Model scale=Base(+)2024.11 | 0.426 | |
| BRAVEnPre-train data=LRS3, Shared params=false, Model scale=Base(+)2024.11 | 0.434 | |
| AV-data2vecPre-train data=LRS3, Shared params=false, Model scale=Base(+)2024.11 | 0.452 | |
| AV-HuBERTPre-train data=LRS3+Vox2, Shared params=false, Model scale=Base(+)2024.11 | 0.461 | |
| RAVEnPre-train data=LRS3, Shared params=false, Model scale=Base(+)2024.11 | 0.47 | |
| VATLMPre-train data=LRS3, Shared params=false, Model scale=Base(+)2024.11 | 0.48 | |
| Lip2VecPre-train data=LRS3, Shared params=false, Model scale=Base(+)2024.11 | 0.495 | |
| AV-HuBERTPre-train data=LRS3, Shared params=false, Model scale=Base(+)2024.11 | 0.518 |