Audio-visual video forgery detection on FakeAVCeleb
99.29AccuracyAV-Lip-Sync+ with FE
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| AV-Lip-Sync+ with FEType=Multimodal (ours), Modality=AV, Class=Real2023.11 | 99.29 | 100 | 98.57 | 99.28 | |
| AV-Lip-Sync+ with FEType=Multimodal (ours), Modality=AV, Class=Fake2023.11 | 99.29 | 98.59 | 100 | 99.29 | |
| FRADEType=Multimodal, Modality=AV2023.11 | 98.6 | — | — | — | |
| AVFFType=Multimodal, Modality=AV2023.11 | 98.6 | — | — | — | |
| AVTENetType=Multimodal-Ensemble, Modality=AV, Class=Real2023.11 | 98.57 | 100 | 97.14 | 98.55 | |
| AVTENetType=Multimodal-Ensemble, Modality=AV, Class=Fake2023.11 | 98.57 | 97.22 | 100 | 98.59 | |
| AV-Lip-Sync+Type=Multimodal (ours), Modality=AV, Class=Real2023.11 | 98.57 | 98.57 | 98.57 | 98.57 | |
| AV-Lip-Sync+Type=Multimodal (ours), Modality=AV, Class=Fake2023.11 | 98.57 | 98.57 | 98.57 | 98.57 | |
| MMMS-BAType=Multimodal, Modality=AV2023.11 | 97.9 | — | — | — | |
| MIS-AVioDDType=Fusion, Modality=AV2023.11 | 96.2 | — | — | — | |
| PVASS-MDDType=Multimodal, Modality=AV2023.11 | 95.7 | — | — | — | |
| Intra-Cross-modalType=Multimodal, Modality=AV2023.11 | 94.59 | — | — | — | |
| AV-Lip-SyncType=Multimodal, Modality=AV, Class=Real2023.11 | 93.57 | 91.78 | 95.71 | 93.71 | |
| AV-Lip-SyncType=Multimodal, Modality=AV, Class=Fake2023.11 | 93.57 | 95.52 | 91.43 | 93.43 | |
| AVFakeNetType=Multimodal-Ensemble, Modality=AV2023.11 | 93.41 | — | — | — | |
| MultimodaltraceType=Multimodal, Modality=AV2023.11 | 92.9 | — | — | — | |
| A-V AnomalyType=Multimodal, Modality=AV2023.11 | 92.71 | — | — | — | |
| Zero-ShotType=Fusion, Modality=AV2023.11 | 91.94 | — | — | — | |
| Ensemble modelType=Multimodal-Ensemble, Modality=AV, Class=Real2023.11 | 89.29 | 83.13 | 98.57 | 90.2 | |
| Ensemble modelType=Multimodal-Ensemble, Modality=AV, Class=Fake2023.11 | 89.29 | 98.25 | 80 | 88.19 | |
| MCLType=Contrastive Learning, Modality=AV2023.11 | 89.25 | — | — | — | |
| POI-ForensicsType=Identity Aware, Modality=AV2023.11 | 85.5 | — | — | — | |
| AVoiD-DFType=Fusion, Modality=AV2023.11 | 83.7 | — | — | — | |
| VGG16Type=Unimodal, Modality=V, Class=Real2023.11 | 81.03 | 69.35 | 89.66 | 78.21 | |
| VGG16Type=Unimodal, Modality=V, Class=Fake2023.11 | 81.03 | 87.24 | 77.5 | 82.08 | |
| VGG16Type=Ensemble (Soft-Voting), Modality=AV, Class=Real2023.11 | 78.04 | 69.35 | 89.66 | 78.21 | |
| VGG16Type=Ensemble (Soft-Voting), Modality=AV, Class=Fake2023.11 | 78.04 | 89.48 | 68.94 | 77.88 | |
| VGG16Type=Ensemble (Hard-Voting), Modality=AV, Class=Real2023.11 | 78.04 | 69.35 | 89.66 | 78.21 | |
| VGG16Type=Ensemble (Hard-Voting), Modality=AV, Class=Fake2023.11 | 78.04 | 89.48 | 68.94 | 77.88 | |
| XceptionType=Unimodal, Modality=A, Class=Real2023.11 | 76.26 | 87.5 | 60.87 | 71.79 | |
| XceptionType=Unimodal, Modality=A, Class=Fake2023.11 | 76.26 | 70.33 | 91.43 | 79.5 | |
| LipForensicsType=Unimodal, Modality=V, Class=Real2023.11 | 76 | 70 | 91 | 80 | |
| LipForensicsType=Unimodal, Modality=V, Class=Fake2023.11 | 76 | 88 | 61 | 72 | |
| MDSType=Multimodal-4, Modality=AV, Class=Real2023.11 | 69.29 | 62.16 | 98.57 | 76.24 | |
| MDSType=Multimodal-4, Modality=AV, Class=Fake2023.11 | 69.29 | 96.55 | 40 | 56.57 | |
| Multimodal-2Type=Multimodal-2, Modality=AV, Class=Real2023.11 | 67.4 | 71 | 58.7 | 64.3 | |
| Multimodal-2Type=Multimodal-2, Modality=AV, Class=Fake2023.11 | 67.4 | 64.8 | 76 | 70 | |
| CDCNType=Multimodal-3, Modality=AV, Class=Real2023.11 | 51.5 | 50 | 6.8 | 12 | |
| CDCNType=Multimodal-3, Modality=AV, Class=Fake2023.11 | 51.5 | 50 | 94 | 65.1 | |
| Multimodal-1Type=Multimodal-1, Modality=AV, Class=Real2023.11 | 50 | 0 | 0 | 0 | |
| Multimodal-1Type=Multimodal-1, Modality=AV, Class=Fake2023.11 | 50 | 49.6 | 100 | 66.3 |