Automatic Lip-Reading on LRS3 v1 (dev)
16.92WERAV-HuBERT + relaxed cross attention
Evaluation results
| Method | Links | |
|---|---|---|
| AV-HuBERT + relaxed cross attentionUnlabeled data (pre-training)=1,326h, Labeled data (fine-tuning)=433h + 1,326h, self-training=true, Language Model=Joint Transformer Decoder, matched inference=false2022.09 | 16.92 | |
| AV-HuBERT + relaxed cross attentionUnlabeled data (pre-training)=1,326h, Labeled data (fine-tuning)=433h + 1,326h, self-training=true, Language Model=None, matched inference=false2022.09 | 17.4 | |
| AV-HuBERT + relaxed cross attentionUnlabeled data (pre-training)=1,326h, Labeled data (fine-tuning)=433h, Language Model=Joint Transformer Decoder, matched inference=false2022.09 | 21.05 | |
| AV-HuBERT (resimulated)Unlabeled data (pre-training)=1,326h, Labeled data (fine-tuning)=433h, Language Model=Joint Transformer Decoder2022.09 | 21.61 | |
| AV-HuBERT (resimulated)Unlabeled data (pre-training)=1,326h, Labeled data (fine-tuning)=433h, Language Model=None2022.09 | 21.9 | |
| AV-HuBERT + relaxed cross attentionUnlabeled data (pre-training)=1,326h, Labeled data (fine-tuning)=433h, Language Model=None, matched inference=false2022.09 | 22.12 | |
| AV-HuBERT + relaxed cross attentionUnlabeled data (pre-training)=1,326h, Labeled data (fine-tuning)=30h, Language Model=Joint Transformer Decoder, matched inference=false2022.09 | 45.11 | |
| AV-HuBERT + relaxed cross attentionUnlabeled data (pre-training)=1,326h, Labeled data (fine-tuning)=30h, Language Model=None, matched inference=false2022.09 | 45.92 | |
| AV-HuBERT + relaxed cross attentionUnlabeled data (pre-training)=1,326h, Labeled data (fine-tuning)=30h, Language Model=Joint Transformer Decoder, matched inference=true2022.09 | 46.46 | |
| AV-HuBERT + relaxed cross attentionUnlabeled data (pre-training)=1,326h, Labeled data (fine-tuning)=30h, Language Model=None, matched inference=true2022.09 | 46.55 | |
| AV-HuBERT + relaxed self-attentionUnlabeled data (pre-training)=1,326h, Labeled data (fine-tuning)=30h, Language Model=Joint Transformer Decoder, matched inference=true2022.09 | 46.68 | |
| AV-HuBERT + smoothed focusUnlabeled data (pre-training)=1,326h, Labeled data (fine-tuning)=30h, Language Model=Joint Transformer Decoder2022.09 | 46.69 | |
| AV-HuBERT + relaxed self-attentionUnlabeled data (pre-training)=1,326h, Labeled data (fine-tuning)=30h, Language Model=None, matched inference=true2022.09 | 46.85 | |
| AV-HuBERT + relaxed self-attentionUnlabeled data (pre-training)=1,326h, Labeled data (fine-tuning)=30h, Language Model=None, matched inference=false2022.09 | 46.9 | |
| AV-HuBERT + relaxed self-attentionUnlabeled data (pre-training)=1,326h, Labeled data (fine-tuning)=30h, Language Model=Joint Transformer Decoder, matched inference=false2022.09 | 46.95 | |
| AV-HuBERT + smoothed focusUnlabeled data (pre-training)=1,326h, Labeled data (fine-tuning)=30h, Language Model=None2022.09 | 47.08 | |
| AV-HuBERT (resimulated)Unlabeled data (pre-training)=1,326h, Labeled data (fine-tuning)=30h, Language Model=None2022.09 | 47.36 | |
| AV-HuBERT (resimulated)Unlabeled data (pre-training)=1,326h, Labeled data (fine-tuning)=30h, Language Model=Joint Transformer Decoder2022.09 | 47.61 |