# DuoGesture: Neuro-Inspired and Biomechanically Informed Dual-Stream Co-Speech Gesture Generation

Source: [DuoGesture: Neuro-Inspired and Biomechanically Informed Dual-Stream Co-Speech Gesture Generation](<https://www.sota2.com/research/paper/6a167aa0dbd7150506f7fff8>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-06T05:00:11.965Z

Authors: Ferdinand Paar, Lanmiao Liu, Asl{\\i} \\"Ozy\\"urek, Serge Thill, Esam Ghaleb

Published: 2026-05-25T18:03:41.000Z

## Abstract

Co-speech gesture generation requires both semantic expressivity and biomechanically plausible rhythmic motion. Existing holistic gesture models mix lexically grounded semantic gestures with frequent prosody-aligned beat gestures. This limits semantic grounding, speech-motion alignment, and kinematic smoothness. We propose \\emph{DuoGesture}, a neuro-inspired and biomechanically informed dual-stream approach that decomposes co-speech gesture synthesis into coupled semantic and beat streams. The two streams are coordinated by a \\emph{Semantic Variational Information Bottleneck}, a stochastic frame-level gate that learns when semantic gestures should override rhythmic beat motion. The semantic stream is controlled by \\emph{Motion-Grounded Semantic Conditioning}, which replaces purely linguistic word embeddings with motion-language representations to provide motion-aligned semantic priors for long-tailed lexical triggers of gestures. The beat stream is further regularised by an \\emph{Inertial Beat Prior}, an anthropometry-weighted arm-chain module that reduces jitter and improves rhythmic consistency without constraining semantic frames. Objective evaluations and subjective experiments show that DuoGesture outperforms strong holistic baselines, while component ablations confirm the complementary roles of semantic grounding, stochastic stream selection, and biomechanical regularisation.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Co-speech gesture generation on BEAT2 All 25 Speakers](<https://www.sota2.com/research/sota/co-speech-gesture-generation-on-beat2-all-25-speakers.md>): Co-speech gesture generation; BEAT2 All 25 Speakers; FGD: 4.081.

- [Co-speech gesture generation on BEAT One Speaker v2 (Speaker 2)](<https://www.sota2.com/research/sota/co-speech-gesture-generation-on-beat-one-speaker-v2-speaker-2.md>): Co-speech gesture generation; BEAT One Speaker v2 (Speaker 2); FGD: 4.101.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2605.26236>)

[PDF](<https://arxiv.org/pdf/2605.26236.pdf>)