# SyncDPO: Enhancing Temporal Synchronization in Video-Audio Joint Generation via Preference Learning Source: [SyncDPO: Enhancing Temporal Synchronization in Video-Audio Joint Generation via Preference Learning]() Collection index: [llms.txt]() Data updated: 2026-05-14T05:00:13.174Z Authors: Xin Cheng, Xihua Wang, Ying Ba, Yuyue Wang, Kaisi Guan, Yinbo Wang, Wenpu Li, Ruihua Song Published: 2026-05-12T14:22:13.000Z ## Abstract Recent advancements in video-audio joint generation have achieved remarkable success in semantic correspondence. However, achieving precise temporal synchronization, which requires fine-grained alignment between audio events and their visual triggers, remains a challenging problem. The post-training method for joint generation is largely dominated by Supervised Fine-Tuning, but the commonly used Mean Squared Error loss provides insufficient penalties for subtle temporal misalignments. Direct Preference Optimization offers an alternative by introducing explicit misaligned counterparts to better improve temporal sensitivity. In this paper we propose a post-training framework SyncDPO, leveraging DPO to improve the temporal sensitivity of V-A joint generation. Conventional DPO pipelines typically depend on costly sampling-and-ranking procedures to construct preference pairs, resulting in substantial computational cost. To improve efficiency, we introduce a suite of on-the-fly rule-based negative construction strategies that distort temporal structures without incurring additional annotation or sampling. We demonstrate that the temporal alignment capability can be effectively reinforced by providing explicit negative supervision through temporally distorted V-A pairs. Accordingly, we implement a curriculum learning strategy that progressively increases the difficulty of negative samples, transitioning from coarse misalignment to subtle inconsistencies. Extensive objective and subjective experiments across four diverse benchmarks, ranging from ambient sound videos to human speech videos, demonstrate that SyncDPO significantly outperforms other methods in improving model's temporal alignment capability. It also demonstrates superior generalization on out-of-distribution benchmark by capturing intrinsic motion-sound dynamics. Demo and code is available in https://syncdpo.github.io/syncdpo/. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Ambient Sound Generation on AVSync15 (test)](): Ambient Sound Generation; AVSync15 (test); DeSync: 0.56. - [Ambient Sound Generation on VABench (test)](): Ambient Sound Generation; VABench (test); DeSync: 0.43. - [Human Speech Generation on LRS2 (test)](): Human Speech Generation; LRS2 (test); LSE-D: 7.83. - [Ambient Sound Generation on Greatest Hits (test)](): Ambient Sound Generation; Greatest Hits (test); DeSync: 0.3. ## Original sources and implementation [Original paper]() [PDF]()