Co-Speech Gesture Video Generation on PATS (test)
14.01DiversityGround Truth
Evaluation results
| Method | Links | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Ground Truth2025.02 | 14.01 | 0 | 1 | 1 | 0 | 95.69 | 5.33 | — | — | — | — | — | — | — | — | — | — | — | |
| Contextual GestureTrain-T=6 days, Infer-T=3 sec2025.02 | 13.13 | 8.76 | 0.99 | 0.54 | 466.43 | 96.53 | 6.12 | — | — | — | — | — | — | — | — | — | — | — | |
| Contextual Gesture + AnimateAnyoneTrain-T=8.5 days, Infer-T=50 sec2025.02 | 13.06 | 6.56 | 0.99 | 0.54 | 477.82 | 95.63 | 6.04 | — | — | — | — | — | — | — | — | — | — | — | |
| S2G-DiffusionTrain-T=5 days, Infer-T=35 sec2025.02 | 10.08 | 10.54 | 0.98 | 0.45 | 493.43 | 94.54 | 5.63 | — | — | — | — | — | — | — | — | — | — | — | |
| EchoMimicV2Infer-T=1200 sec2025.02 | 9.85 | 13.65 | 0.98 | 0.45 | 466.84 | 95.65 | 5.98 | — | — | — | — | — | — | — | — | — | — | — | |
| ANGIETrain-T=5 days, Infer-T=30 sec2025.02 | 7.87 | 34.13 | 0.78 | 0.37 | 515.43 | 86.32 | 4.98 | — | — | — | — | — | — | — | — | — | — | — | |
| LMDD2026.06 | 7.159 | — | — | — | 54.456 | — | — | — | — | — | 1.115 | 28.835 | 90.5 | 0.027 | 19.246 | 5.027 | 10.257 | 1.531 | |
| LMDDAblation=w/o Appearance2026.06 | 6.511 | — | — | — | 90.955 | — | — | — | — | — | 1.694 | 26.299 | 86.9 | 0.043 | 28.194 | 6.887 | 11.26 | 0.721 | |
| LMDDAblation=w/o attention map2026.06 | 6.405 | — | — | — | 133.441 | — | — | — | — | — | 1.5 | 25.801 | 86.3 | 0.047 | 29.148 | 6.597 | 11.78 | 0.296 | |
| S2G-MDDiffusion2026.06 | 6.28 | — | — | — | 109.424 | — | — | — | — | — | 1.333 | 28.5 | 89.4 | 0.04 | 25.325 | 6.61 | 11.01 | 0.971 | |
| LMDDAblation=w/o mamba-g2026.06 | 6.139 | — | — | — | 70.773 | — | — | — | — | — | — | — | — | — | 31.136 | 6.697 | 11.038 | 1.23 | |
| Ground Truth (GT)2024.04 | 5.911 | 8.976 | 0.1506 | — | 1,852.86 | — | — | 4.76 | 4.77 | 4.73 | — | — | — | — | — | — | — | — | |
| LMDDAblation=w/o Trans. in DIT2026.06 | 5.8 | — | — | — | 153.641 | — | — | — | — | — | 1.576 | 24.79 | 85.4 | 0.057 | 32.234 | 7.213 | 11.692 | 0.309 | |
| S2G-MDDiffusion2024.04 | 5.632 | 18.131 | 0.1273 | — | 2,058.19 | — | — | 3.79 | 3.9 | 3.77 | — | — | — | — | — | — | — | — | |
| TANGO2026.06 | 5.217 | — | — | — | 168.313 | — | — | — | — | — | — | — | — | — | 42.863 | 1.883 | 10.365 | 1.736 | |
| MM-Diffusion2024.04 | 5.189 | 41.626 | 0.1098 | — | 2,656.06 | — | — | 1.77 | 1.69 | 1.47 | — | — | — | — | — | — | — | — | |
| ANGIE2024.04 | 5.089 | 55.655 | 0.1504 | — | 2,965.29 | — | — | 2.07 | 2.19 | 2 | — | — | — | — | — | — | — | — | |
| LMDDAblation=w/o mamba-l2026.06 | 4.706 | — | — | — | 71.032 | — | — | — | — | — | — | — | — | — | 30.587 | 5.593 | 11 | 1.09 | |
| Ground Truth (GT)2024.04 | 4.7 | — | — | — | — | — | — | 4.76 | 4.77 | 4.73 | — | — | — | — | — | — | — | — | |
| MM-DiffusionTrain-T=14 days, Infer-T=600 sec2025.02 | 4.32 | 67.56 | 0.65 | 0.11 | — | 77.65 | 4.14 | — | — | — | — | — | — | — | — | — | — | — | |
| GT2022.12 | 3.97 | — | — | — | — | — | — | 4.29 | 4.36 | — | — | — | — | — | — | — | — | — | |
| ANGIE2022.12 | 3.95 | — | — | — | — | — | — | 4.08 | 4.11 | — | — | — | — | — | — | — | — | — | |
| S2G-MDDiffusionInputs=Audio and initial frame image, Sub-clip length=3.2s2024.04 | 3.91 | — | — | — | — | — | — | 3.79 | 3.9 | 3.77 | — | — | — | — | — | — | — | — | |
| SDT2022.12 | 3.88 | — | — | — | — | — | — | 4.01 | 3.97 | — | — | — | — | — | — | — | — | — | |
| HA2G2022.12 | 3.31 | — | — | — | — | — | — | 3.92 | 4.01 | — | — | — | — | — | — | — | — | — | |
| TriCon2022.12 | 3.02 | — | — | — | — | — | — | 3.74 | 3.85 | — | — | — | — | — | — | — | — | — | |
| ANGIEInputs=Audio and initial frame image2024.04 | 2.53 | — | — | — | — | — | — | 2.07 | 2.19 | 2 | — | — | — | — | — | — | — | — | |
| S2G2022.12 | 2.49 | — | — | — | — | — | — | 3.27 | 3.48 | — | — | — | — | — | — | — | — | — | |
| MM-DiffusionGeneration scheme=Generating 1.6s segments for concatenation (MM-Diffusion-C)2024.04 | 2.02 | — | — | — | — | — | — | 1.77 | 1.69 | 1.47 | — | — | — | — | — | — | — | — | |
| MM-DiffusionGeneration scheme=Directly sampling long noise (MM-Diffusion-D)2024.04 | 1.98 | — | — | — | — | — | — | 1.63 | 1.54 | 1.46 | — | — | — | — | — | — | — | — |