Talking Face Generation on LRW (test)
1SSIMGround Truth
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Ground Truth2021.04 | 1 | — | 0 | 0.173 | 6.5 | — | — | — | — | — | |
| Ground Truth2023.03 | 1 | — | — | — | — | 6.88 | 0.8851 | — | — | — | |
| GTtype=Ground Truth2023.05 | 1 | 100 | 0 | — | — | — | — | — | 7.65 | 1 | |
| LipGAN2020.03 | 0.96 | 33.4 | 0.6 | — | — | — | — | — | — | — | |
| SyncTalkFacetraining_data=whole LRS2 dataset (224 hours)2023.03 | 0.893 | 33.13 | — | — | — | 6.62 | — | — | — | — | |
| Zhou et al.2020.03 | 0.884 | 26.8 | — | — | — | — | — | — | — | — | |
| StyleSyncmode=personalized2023.05 | 0.88 | 32.66 | 0.86 | — | — | — | — | — | 6.35 | 0.93 | |
| Wav2Lip2023.03 | 0.874 | 31.52 | — | — | — | 7.18 | 0.5998 | — | — | — | |
| TalkLip (g)embedding=global audio embedding2023.03 | 0.871 | 30.78 | — | — | — | 7.01 | 0.8657 | — | — | — | |
| TalkLip (l)embedding=local audio embedding2023.03 | 0.867 | 31.24 | — | — | — | 6.44 | 0.7978 | — | — | — | |
| TalkLip (l + c)embedding=local audio embedding, contrastive_learning=true2023.03 | 0.867 | 31.52 | — | — | — | 6.51 | 0.8317 | — | — | — | |
| TalkLip (g + c)embedding=global audio embedding, contrastive_learning=true2023.03 | 0.866 | 31.18 | — | — | — | 7.28 | 0.8781 | — | — | — | |
| Base w.o. L_lipBase=TalkLip (l), ablation=without L_lip loss2023.03 | 0.865 | 31.22 | — | — | — | 6.01 | 0.4858 | — | — | — | |
| Base w.o. L_lip,ganBase=TalkLip (l), ablation=without L_lip loss and GAN loss2023.03 | 0.864 | 30.64 | — | — | — | 5.03 | 0.308 | — | — | — | |
| Wav2Lip2021.04 | 0.862 | — | 5.73 | 0.152 | 6.9 | — | — | — | — | — | |
| PC-AVSpose_setting=pose-controllable2021.04 | 0.861 | — | 3.93 | 0.185 | 6.4 | — | — | — | — | — | |
| Faceformer2023.03 | 0.856 | 29.19 | — | — | — | 5.58 | 0.5343 | — | — | — | |
| StyleSyncmode=generalized2023.05 | 0.85 | 31.78 | 1.18 | — | — | — | — | — | 7.25 | 0.89 | |
| PC-AVSpose_setting=fixed2021.04 | 0.815 | — | 6.14 | 0.18 | 6.3 | — | — | — | — | — | |
| ATVG2021.04 | 0.81 | — | 5.25 | 0.102 | 4.1 | — | — | — | — | — | |
| Wav2Lip-Htraining setting=reproduced2023.05 | 0.8 | 31.38 | 1.2 | — | — | — | — | — | 7.19 | 0.88 | |
| MakeitTalk2021.04 | 0.796 | — | 7.13 | 0.161 | 3.1 | — | — | — | — | — | |
| ATVGnet2023.03 | 0.791 | 30.71 | — | — | — | 5.64 | 0.181 | — | — | — | |
| PC-AVS2023.05 | 0.79 | 30.26 | 1.84 | — | — | — | — | — | 7.19 | 0.82 | |
| Wav2Lip2023.05 | 0.79 | 30.54 | 1.28 | — | — | — | — | — | 7.39 | 0.9 | |
| PC-AVStraining_data=whole LRS2 dataset (224 hours)2023.03 | 0.778 | 30.44 | — | — | — | 6.42 | — | — | — | — | |
| MakeitTalk2023.05 | 0.69 | 29.83 | 2.75 | — | — | — | — | — | 3.88 | 0.79 | |
| Chung et al.2020.03 | 0.46 | 28.06 | 2.22 | — | — | — | — | — | — | — | |
| ATVGnet2023.03 | — | — | — | — | — | — | — | 8.56 | — | — | |
| Faceformer2023.03 | — | — | — | — | — | — | — | 8 | — | — | |
| Ground Truth2023.03 | — | — | — | — | — | — | — | 6.97 | — | — | |
| PC-AVSTraining Data=whole LRS2 dataset (224 hours)2023.03 | — | — | — | — | — | — | — | 7.34 | — | — | |
| SyncTalkFaceTraining Data=whole LRS2 dataset (224 hours)2023.03 | — | — | — | — | — | — | — | 6.97 | — | — | |
| TalkLip (g + c)variant=global encoder + contrastive, Training Data=LRS2 dataset (29 hours)2023.03 | — | — | — | — | — | — | — | 6.51 | — | — | |
| TalkLip (g)variant=global encoder, Training Data=LRS2 dataset (29 hours)2023.03 | — | — | — | — | — | — | — | 6.75 | — | — | |
| TalkLip (l + c)variant=local encoder + contrastive, Training Data=LRS2 dataset (29 hours)2023.03 | — | — | — | — | — | — | — | 7 | — | — | |
| TalkLip (l)variant=local encoder, Training Data=LRS2 dataset (29 hours)2023.03 | — | — | — | — | — | — | — | 7 | — | — | |
| Wav2Lip2023.03 | — | — | — | — | — | — | — | 7.01 | — | — |