# Text Dictates, Music Decorates: Energy-based Attention for Editable Dance Motion Generation

Source: [Text Dictates, Music Decorates: Energy-based Attention for Editable Dance Motion Generation](<https://www.sota2.com/research/paper/6a3a1364206542251d7c7433>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-07-08T05:00:04.940Z

Authors: Seong Jong Yoo, Siyuan Peng, Felix Gu, Stratis Aloimonos, Cornelia Ferm\\"uller

Published: 2026-06-22T00:01:05.000Z

## Abstract

Choreographic motion generation poses unique challenges for AI, demanding precise semantic control over complex, temporally structured, and expressive full-body dynamics. While existing models can synthesize motion from music, they remain largely black boxes. Conversely, attempting to condition generation on both text and music frequently leads to modality collapse, where dense acoustic rhythms overwhelm sparse semantic text prompts, destroying user controllability. To resolve this spatial-temporal conflict, we propose STREAM (Structural-Temporal Rhythmic Energy-based Attention for Motion), a modality-decoupled diffusion transformer. STREAM strictly separates conditioning pathways: global text semantics dictate the kinematic structure via Adaptive Layer Normalization (AdaLN), while a novel Bimodal Energy-Based Attention Module (BEAM) routes these features to the musical beat without overwriting the semantics. We further introduce Motorica++, a newly curated dataset enriched with domain-specific dance vocabulary and frame-level semantic annotations from existing Motorica dataset. Additionally, to rigorously quantify zero-shot editability, we propose the Exchange Evaluation Protocol and Editable Dance Score (EDS). Through extensive experiments, STREAM achieves state-of-the-art alignment between motion and music while perfectly preserving choreographic semantics, positioning AI not merely as a reactive synthesizer, but as a controllable, collaborative partner for artistic direction. The source code and datasets are available at https://github.com/SeongJong-Yoo/STREAM.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Dance Generation on Motorica++ (test)](<https://www.sota2.com/research/sota/dance-generation-on-motorica-plus-plus-test.md>): Dance Generation; Motorica++ (test); Distk: 8.67.

- [Dance Generation on AIST++ (test)](<https://www.sota2.com/research/sota/dance-generation-on-aist-plus-plus-test.md>): Dance Generation; AIST++ (test); Distance (k): 8.74.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2606.22726>)

[PDF](<https://arxiv.org/pdf/2606.22726.pdf>)
