# VECL-TTS: Voice identity and Emotional style controllable Cross-Lingual Text-to-Speech

Source: [VECL-TTS: Voice identity and Emotional style controllable Cross-Lingual Text-to-Speech](<https://www.sota2.com/research/paper/68fa739c1e209131cea5d960>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-27T06:35:02.634Z

Authors: Ashishkumar Gudmalwar, Nirmesh Shah, Sai Akarsh, Pankaj Wasnik, Rajiv Ratn Shah

Published: 2024-06-12T10:51:29.000Z

## Abstract

Despite the significant advancements in Text-to-Speech (TTS) systems, their full utilization in automatic dubbing remains limited. This task necessitates the extraction of voice identity and emotional style from a reference speech in a source language and subsequently transferring them to a target language using cross-lingual TTS techniques. While previous approaches have mainly concentrated on controlling voice identity within the cross-lingual TTS framework, there has been limited work on incorporating emotion and voice identity together. To this end, we introduce an end-to-end Voice Identity and Emotional Style Controllable Cross-Lingual (VECL) TTS system using multilingual speakers and an emotion embedding network. Moreover, we introduce content and style consistency losses to enhance the quality of synthesized speech further. The proposed system achieved an average relative improvement of 8.83\\% compared to the state-of-the-art (SOTA) methods on a database comprising English and three Indian languages (Hindi, Telugu, and Marathi).

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Cross-speaker style transfer on ESD (test)](<https://www.sota2.com/research/sota/cross-speaker-style-transfer-on-esd-test.md>): Cross-speaker Style Transfer; ESD (test); nMOS: 2.707.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2406.08076>)

[PDF](<https://arxiv.org/pdf/2406.08076.pdf>)
