# ReLyMe: Improving Lyric-to-Melody Generation by Incorporating Lyric-Melody Relationships

Source: [ReLyMe: Improving Lyric-to-Melody Generation by Incorporating Lyric-Melody Relationships](<https://www.sota2.com/research/paper/68fa6d001e209131cea37366>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T00:23:36.215Z

Authors: Chen Zhang, Luchin Chang, Songruoyao Wu, Xu Tan, Tao Qin, Tie-Yan Liu, Kejun Zhang

Published: 2022-07-12T17:09:44.000Z

## Abstract

Lyric-to-melody generation, which generates melody according to given lyrics, is one of the most important automatic music composition tasks. With the rapid development of deep learning, previous works address this task with end-to-end neural network models. However, deep learning models cannot well capture the strict but subtle relationships between lyrics and melodies, which compromises the harmony between lyrics and generated melodies. In this paper, we propose ReLyMe, a method that incorporates Relationships between Lyrics and Melodies from music theory to ensure the harmony between lyrics and melodies. Specifically, we first introduce several principles that lyrics and melodies should follow in terms of tone, rhythm, and structure relationships. These principles are then integrated into neural network lyric-to-melody models by adding corresponding constraints during the decoding process to improve the harmony between lyrics and melodies. We use a series of objective and subjective metrics to evaluate the generated melodies. Experiments on both English and Chinese song datasets show the effectiveness of ReLyMe, demonstrating the superiority of incorporating lyric-melody relationships from the music domain into neural lyric-to-melody generation.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Lyric-to-Melody generation on GTSinger (test)](<https://www.sota2.com/research/sota/lyric-to-melody-generation-on-gtsinger-test.md>): Lyric-to-Melody generation; GTSinger (test); MOS: 3.26.

- [Lyric-to-Melody generation on GTSinger English (test)](<https://www.sota2.com/research/sota/lyric-to-melody-generation-on-gtsinger-english-test.md>): Lyric-to-Melody generation; GTSinger English (test); Pitch Deviation (PD): 31.27.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2207.05688>)

[PDF](<https://arxiv.org/pdf/2207.05688.pdf>)