Talking Face Generation on LRS2 (test)
1SSIMGround Truth
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Ground Truth2023.03 | 1 | — | — | — | 8.25 | 23.82 | 0.409 | — | — | — | — | — | — | |
| IP_LAP2023.05 | 0.9399 | — | — | 32.91 | — | — | — | — | 0.0303 | 27.87 | 0.0129 | 0.6523 | — | |
| Wav2lip2023.05 | 0.8962 | — | — | 27.92 | — | — | — | — | 0.0741 | 43.46 | 0.02 | 0.5925 | — | |
| SyncTalkFacetraining_data=whole LRS2 dataset (224 hours)2023.03 | 0.876 | — | — | 32.59 | 7.93 | — | — | — | — | — | — | — | — | |
| Wav2Lip2023.03 | 0.854 | — | — | 31.36 | 8.4 | 82.06 | 0.739 | — | — | — | — | — | — | |
| TalkLip (g)embedding=global audio embedding2023.03 | 0.854 | — | — | 30.86 | 8.38 | 25.31 | 0.365 | — | — | — | — | — | — | |
| Base w.o. L_lipBase=TalkLip (l), ablation=without L_lip loss2023.03 | 0.852 | — | — | 31.08 | 7.09 | 103.57 | 0.822 | — | — | — | — | — | — | |
| Base w.o. L_lip,ganBase=TalkLip (l), ablation=without L_lip loss and GAN loss2023.03 | 0.851 | — | — | 30.7 | 5.93 | 116.26 | 0.893 | — | — | — | — | — | — | |
| TalkLip (l + c)embedding=local audio embedding, contrastive_learning=true2023.03 | 0.85 | — | — | 31.14 | 7.76 | 38 | 0.492 | — | — | — | — | — | — | |
| TalkLip (g + c)embedding=global audio embedding, contrastive_learning=true2023.03 | 0.85 | — | — | 31.19 | 8.53 | 23.43 | 0.351 | — | — | — | — | — | — | |
| TalkLip (l)embedding=local audio embedding2023.03 | 0.849 | — | — | 31.38 | 7.58 | 45.74 | 0.557 | — | — | — | — | — | — | |
| Faceformer2023.03 | 0.84 | — | — | 29.47 | 6.42 | 97.64 | 0.79 | — | — | — | — | — | — | |
| ATVGnet2023.03 | 0.751 | — | — | 30.42 | 5.05 | 113.69 | 0.918 | — | — | — | — | — | — | |
| PC-AVStraining_data=whole LRS2 dataset (224 hours)2023.03 | 0.747 | — | — | 29.89 | 6.73 | — | — | — | — | — | — | — | — | |
| MakeItTalk2023.05 | 0.5562 | — | — | 17.25 | — | — | — | — | 0.2237 | 76.57 | 0.0502 | 0.5799 | — | |
| PC-AVS2023.05 | 0.4867 | — | — | 15.75 | — | — | — | — | 0.2802 | 110.6 | 0.0757 | 0.3927 | — | |
| EAMM2023.05 | 0.4623 | — | — | 15.17 | — | — | — | — | 0.3398 | 91.95 | 0.1519 | 0.2318 | — | |
| ATVGnet2023.05 | 0.3944 | — | — | 11.55 | — | — | — | — | 0.5575 | 223.26 | 0.274 | 0.102 | — | |
| ATVGnet2023.03 | — | — | — | — | — | — | — | 8.65 | — | — | — | — | — | |
| Chung et al.2020.03 | — | 2.95 | 3.1 | — | — | — | — | — | — | — | — | — | — | |
| Faceformer2023.03 | — | — | — | — | — | — | — | 7.8 | — | — | — | — | — | |
| Ground Truth2023.03 | — | — | — | — | — | — | — | 6.45 | — | — | — | — | — | |
| LipGAN2020.03 | — | 3.68 | 3.73 | — | — | — | — | — | — | — | — | — | — | |
| PC-AVSTraining Data=whole LRS2 dataset (224 hours)2023.03 | — | — | — | — | — | — | — | 7.3 | — | — | — | — | — | |
| SyncTalkFaceTraining Data=whole LRS2 dataset (224 hours)2023.03 | — | — | — | — | — | — | — | 6.26 | — | — | — | — | — | |
| TalkLip (g + c)variant=global encoder + contrastive, Training Data=LRS2 dataset (29 hours)2023.03 | — | — | — | — | — | — | — | 6 | — | — | — | — | — | |
| TalkLip (g)variant=global encoder, Training Data=LRS2 dataset (29 hours)2023.03 | — | — | — | — | — | — | — | 6.01 | — | — | — | — | — | |
| TalkLip (l + c)variant=local encoder + contrastive, Training Data=LRS2 dataset (29 hours)2023.03 | — | — | — | — | — | — | — | 6.56 | — | — | — | — | — | |
| TalkLip (l)variant=local encoder, Training Data=LRS2 dataset (29 hours)2023.03 | — | — | — | — | — | — | — | 6.63 | — | — | — | — | — | |
| Wav2Lip2023.03 | — | — | — | — | — | — | — | 6.58 | — | — | — | — | — | |
| Zhou et al.2020.03 | — | 2.41 | 2.42 | — | — | — | — | — | — | — | — | — | — |