# DuplexOmni: Real-Time Listening, Seeing, Thinking, and Speaking for Full-Duplex Interaction

Source: [DuplexOmni: Real-Time Listening, Seeing, Thinking, and Speaking for Full-Duplex Interaction](<https://www.sota2.com/research/paper/6a279e5690966ade919bf9e4>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-10T05:15:22.625Z

Authors: Muye Huang, Lingling Zhang, Xingyu Yu, Lei Shi, Zhanyu Ma, Jun Xu, Jiuchong Gao, Jinghua Hao, Renqing He, Jun Liu

Published: 2026-06-08T08:23:02.000Z

## Abstract

Human interaction is continuous, multimodal, and full-duplex by nature. Although recent omni models have made substantial progress in unified speech, vision, and text modeling, combining seamless real-time interaction with complex reasoning and tool use remains challenging. We present DuplexOmni, a method for real-time multimodal full-duplex interaction. DuplexOmni separates model capability into an interaction layer and a thinking layer, which collaborate asynchronously in parallel. The interaction layer is implemented by the DuplexOmni model, an end-to-end system that processes streaming audio and video inputs while generating text and speech responses in real time. The thinking layer is a pluggable module that provides complex reasoning and tool-use capabilities. To support this method, we further develop a Writer-Director pipeline for constructing continuous-interaction training data. Experiments show that DuplexOmni achieves strong performance on multiple public benchmarks and exhibits natural full-duplex interaction ability.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Audio Understanding on Big Bench Audio](<https://www.sota2.com/research/sota/audio-understanding-on-big-bench-audio.md>): Audio Understanding; Big Bench Audio; Audio Accuracy: 77.2.

- [Streaming Interaction on Daily-Omni](<https://www.sota2.com/research/sota/streaming-interaction-on-daily-omni.md>): Streaming Interaction; Daily-Omni; Accuracy: 53.8.

- [Automatic Speech Recognition on LibriSpeech (WER)](<https://www.sota2.com/research/sota/automatic-speech-recognition-on-librispeech-wer.md>): Automatic Speech Recognition; LibriSpeech; WER: 11.92.

- [Duplex Dialogue Turn-Taking on Full-Duplex-Bench](<https://www.sota2.com/research/sota/duplex-dialogue-turn-taking-on-full-duplex-bench.md>): Duplex Dialogue Turn-Taking; Full-Duplex-Bench; Synthetic TOR for Pause Handling: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2606.09186>)

[PDF](<https://arxiv.org/pdf/2606.09186.pdf>)