List your company

Audio-driven human video generation on Combined Author dataset + PATS (test)

0.8275SSIMOurs
0.5385880.6135940.68860.763606Mar 25, 2025

Evaluation results

MethodLinks
OursProtocol=Zero-shot2025.030.82750.136845.4137.45527.240.23550.94850.88320.34010.7454
w/o R2-DiTRegional Refinement DiT=false2025.030.81180.139948.2737.57549.010.2330.94280.85790.33190.7248
w/o MTMotion Transfer=false2025.030.80230.179545.7537.52629.30.23430.94240.82830.32640.7376
w/o HPEHead Pose Estimation=false2025.030.79990.141650.3735.13660.130.2320.93190.85260.32780.7331
w/o IAIdentity-Aware=false2025.030.76260.145448.6937.59575.940.2320.94190.84340.33760.6839
w/o AAAudio-Aware=false2025.030.72530.192763.9435.11667.290.23090.94570.84080.33950.7016
Talk-CNxtGesture synthesis=TalkShow, Video generation=ControlNext2025.030.6670.30280.4985.441,289.060.21870.93640.76330.41270.4887
Prob-CNxtGesture synthesis=ProbTalk, Video generation=ControlNext2025.030.6630.296674.262.651,420.410.2260.93520.86770.46420.616
Prob-MimicGesture synthesis=ProbTalk, Video generation=MimicMotion2025.030.55460.44131.25112.851,563.950.22970.91720.86910.43650.3265
Talk-MimicGesture synthesis=TalkShow, Video generation=MimicMotion2025.030.54970.4375136.82134.021,648.920.22570.89760.74650.34670.419