# Conversational Human Audio-visual Talking Dialogue Generation

Source: [Conversational Human Audio-visual Talking Dialogue Generation](<https://www.sota2.com/research/paper/6a4c87f985dba69fef8b2564>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-07-08T05:00:04.941Z

Authors: Junhao Song, Lluis Guasch, Xilin He, Zhongyu Yang, Yingfang Yuan, Weicheng Xie, Linlin Shen, Haijun Lin, Shizhe Liu, Wei Pang, Siyang Song

Published: 2026-07-02T22:17:55.000Z

## Abstract

Large-scale dyadic interactive audio-visual dialogue (DIAD) datasets provide fundamental data resources for developing humanoid interactive virtual agents and digital humans. However, collecting such data is time-consuming, expensive, and ethically sensitive. To address this, we propose CHAT, a new dyadic interactive audio-visual dialogue generation (DIADG) framework that generates diverse, paired, and mutually responsive speech-face dialogue clips from a single textual prompt. CHAT unifies large language models and talking face models with interactive audio and facial behaviour refinement modules, enabling the generation of aligned dyadic dialogue clips with diverse contents and facial identities. Experiments show that CHAT outperforms existing related methods designed for similar tasks under both objective and subjective evaluations. Moreover, our synthesised CHAT-AVD-50k dataset serves as effective pre-training data for downstream interactive head generation, consistently improving PerFRDiff and ReactDiff on REACT 2024. CHAT offers a scalable alternative to the costly and ethically sensitive collection of real dyadic interaction data.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Audio-visual Dialogue Generation on DIADG evaluation (mean of 1000 samples)](<https://www.sota2.com/research/sota/audio-visual-dialogue-generation-on-diadg-evaluation-mean-of-1000-samples.md>): Audio-visual dialogue generation; DIADG evaluation (mean of 1000 samples); FID: 17.33.

- [Dialogue-driven Audio-visual Dialogue Generation on DIADG User Study (evaluation)](<https://www.sota2.com/research/sota/dialogue-driven-audio-visual-dialogue-generation-on-diadg-user-study-evaluation.md>): Dialogue-driven Audio-visual Dialogue Generation; DIADG User Study (evaluation); Visual Score: 4.6.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2607.02799>)

[PDF](<https://arxiv.org/pdf/2607.02799.pdf>)