# Gelina: Unified Speech and Gesture Synthesis via Interleaved Token Prediction

Source: [Gelina: Unified Speech and Gesture Synthesis via Interleaved Token Prediction](<https://www.sota2.com/research/paper/68fa7c1e1e209131cea8c830>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-30T05:00:04.703Z

Authors: T\\'eo Guichoux, Th\\'eodor Lemerle, Shivam Mehta, Jonas Beskow, Gustav Eje Henter, Laure Soulier, Catherine Pelachaud, Nicolas Obin

Published: 2025-10-13T09:51:26.000Z

## Abstract

Human communication is multimodal, with speech and gestures tightly coupled, yet most computational methods for generating speech and gestures synthesize them sequentially, weakening synchrony and prosody alignment. We introduce Gelina, a unified framework that jointly synthesizes speech and co-speech gestures from text using interleaved token sequences in a discrete autoregressive backbone, with modality-specific decoders. Gelina supports multi-speaker and multi-style cloning and enables gesture-only synthesis from speech inputs. Subjective and objective evaluations demonstrate competitive speech quality and improved gesture generation over unimodal baselines.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Co-speech gesture generation on BEAT All Speakers 2](<https://www.sota2.com/research/sota/co-speech-gesture-generation-on-beat-all-speakers-2.md>): Co-speech gesture generation; BEAT All Speakers 2; BC: 0.824.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2510.12834>)

[PDF](<https://arxiv.org/pdf/2510.12834.pdf>)