Automatic Visual Speech Recognition on LRS2 clean (test)
2.2WERMIR-GAN
Evaluation results
| Method | Links | |
|---|---|---|
| MIR-GANBackbone=Transformer, Criterion=S2S, Unlabeled data (hrs)=1759, Labeled data (hrs)=224, DataAug=true, LM=false, Learning Strategy=Proposed (Self-Supervised)2023.06 | 2.2 | |
| Base modelBackbone=Transformer, Criterion=S2S, Unlabeled data (hrs)=1759, Labeled data (hrs)=224, DataAug=true, LM=false, Learning Strategy=Proposed (Self-Supervised)2023.06 | 2.3 | |
| MoCo+wav2vecBackbone=Transformer, Criterion=S2S + CTC, Unlabeled data (hrs)=60000, Labeled data (hrs)=381, DataAug=true, LM=false, Learning Strategy=Self-Supervised2023.06 | 2.6 | |
| MIR-GANBackbone=Conformer, Criterion=S2S, Unlabeled data (hrs)=0, Labeled data (hrs)=224, DataAug=true, LM=false, Learning Strategy=Proposed (Supervised)2023.06 | 3.2 | |
| Hyb-ConformerBackbone=Conformer, Criterion=S2S + CTC, Unlabeled data (hrs)=0, Labeled data (hrs)=381, DataAug=true, LM=true, Learning Strategy=Supervised2023.06 | 3.7 | |
| Base modelBackbone=Conformer, Criterion=S2S, Unlabeled data (hrs)=0, Labeled data (hrs)=224, DataAug=true, LM=false, Learning Strategy=Proposed (Supervised)2023.06 | 3.9 | |
| MIR-GANBackbone=Transformer, Criterion=S2S, Unlabeled data (hrs)=0, Labeled data (hrs)=224, DataAug=true, LM=false, Learning Strategy=Proposed (Supervised)2023.06 | 4.5 | |
| Base modelBackbone=Transformer, Criterion=S2S, Unlabeled data (hrs)=0, Labeled data (hrs)=224, DataAug=true, LM=false, Learning Strategy=Proposed (Supervised)2023.06 | 5.4 | |
| LF-MMI TDNNBackbone=TDNN, Criterion=LF-MMI, Unlabeled data (hrs)=0, Labeled data (hrs)=224, DataAug=true, LM=false, Learning Strategy=Supervised2023.06 | 5.9 | |
| Hyb-RNNBackbone=RNN, Criterion=S2S + CTC, Unlabeled data (hrs)=0, Labeled data (hrs)=397, DataAug=true, LM=true, Learning Strategy=Supervised2023.06 | 7 | |
| TM-CTCBackbone=Transformer, Criterion=CTC, Unlabeled data (hrs)=0, Labeled data (hrs)=1519, DataAug=true, LM=true, Learning Strategy=Supervised2023.06 | 8.2 | |
| TM-seq2seqBackbone=Transformer, Criterion=S2S, Unlabeled data (hrs)=0, Labeled data (hrs)=1519, DataAug=true, LM=true, Learning Strategy=Supervised2023.06 | 8.5 |