# Tora3: Trajectory-Guided Audio-Video Generation with Physical Coherence Source: [Tora3: Trajectory-Guided Audio-Video Generation with Physical Coherence]() Collection index: [llms.txt]() Data updated: 2026-04-18T05:00:04.772Z Authors: Junchao Liao, Zhenghao Zhang, Xiangyu Meng, Litao Li, Ziying Zhang, Siyu Zhu, Long Qin, Weizhi Wang Published: 2026-04-10T07:37:03.000Z ## Abstract Audio-video (AV) generation has recently made strong progress in perceptual quality and multimodal coherence, yet generating content with plausible motion-sound relations remains challenging. Existing methods often produce object motions that are visually unstable and sounds that are only loosely aligned with salient motion or contact events, largely because they lack an explicit motion-aware structure shared by video and audio generation. We present Tora3, a trajectory-guided AV generation framework that improves physical coherence by using object trajectories as a shared kinematic prior. Rather than treating trajectories as a video-only control signal, Tora3 uses them to jointly guide visual motion and acoustic events. Specifically, we design a trajectory-aligned motion representation for video, a kinematic-audio alignment module driven by trajectory-derived second-order kinematic states, and a hybrid flow matching scheme that preserves trajectory fidelity in trajectory-conditioned regions while maintaining local coherence elsewhere. We further curate PAV, a large-scale AV dataset emphasizing motion-relevant patterns with automatically extracted motion annotations. Extensive experiments show that Tora3 improves motion realism, motion-sound synchronization, and overall AV generation quality over strong open-source baselines. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Event-level Audio Timing on User Study](): Event-level Audio Timing; User Study; Win Rate: 65.9. - [Motion-Sound Intensity Alignment on User Study](): Motion-Sound Intensity Alignment; User Study; Win Rate: 66.1. - [Overall Preference on User Study](): Overall Preference; User Study; Win Rate: 63.9. - [Video Quality on User Study](): Video Quality; User Study; Win Rate: 59.2. - [Motion Alignment on User Study](): Motion Alignment; User Study; Win Rate: 45.1. - [Motion-conditioned Audio-Video Generation on Audio-Video Generation Evaluation Set](): Motion-conditioned Audio-Video Generation; Audio-Video Generation Evaluation Set; AS: 4.61. ## Original sources and implementation [Original paper]() [PDF]()