# PROST-LLM: Progressively Enhancing the Speech-to-Speech Translation Capability in LLMs

Source: [PROST-LLM: Progressively Enhancing the Speech-to-Speech Translation Capability in LLMs](<https://www.sota2.com/research/paper/6976f4e6a2b28e761d87c0c7>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-27T05:00:04.746Z

Authors: Jing Xu, Jiaqi Wang, Daxin Tan, Xiao Chen

Published: 2026-01-23T10:16:13.000Z

## Abstract

Although Large Language Models (LLMs) excel in many tasks, their application to Speech-to-Speech Translation (S2ST) is underexplored and hindered by data scarcity. To bridge this gap, we propose PROST-LLM (PROgressive Speech-to-speech Translation) to enhance the S2ST capabilities in LLMs progressively. First, we fine-tune the LLMs with the CVSS corpus, employing designed tri-task learning and chain of modality methods to boost the initial performance. Then, leveraging the fine-tuned model, we generate preference pairs through self-sampling and back-translation without human evaluation. Finally, these preference pairs are used for preference optimization to enhance the model's S2ST capability further. Extensive experiments confirm the effectiveness of our proposed PROST-LLM in improving the S2ST capability of LLMs.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Speech Naturalness Assessment on CVSS-C](<https://www.sota2.com/research/sota/speech-naturalness-assessment-on-cvss-c.md>): Speech Naturalness Assessment; CVSS-C; UTMOS Score: 3.74.

- [Speech Naturalness Assessment on CVSS-T](<https://www.sota2.com/research/sota/speech-naturalness-assessment-on-cvss-t.md>): Speech Naturalness Assessment; CVSS-T; UTMOS: 3.67.

- [Speech-to-Text Translation (en2fra) on CVSS-C](<https://www.sota2.com/research/sota/speech-to-text-translation-en2fra-on-cvss-c.md>): Speech-to-Text Translation (en2fra); CVSS-C; BLEU: 29.97.

- [Speech-to-Speech Translation (en2fra) on CVSS-C](<https://www.sota2.com/research/sota/speech-to-speech-translation-en2fra-on-cvss-c.md>): Speech-to-Speech Translation (en2fra); CVSS-C; BLEU: 25.12.

- [Speech-to-Speech Translation (en2fra) on CVSS-T](<https://www.sota2.com/research/sota/speech-to-speech-translation-en2fra-on-cvss-t.md>): Speech-to-Speech Translation (en2fra); CVSS-T; BLEU Score: 22.54.

- [Speech-to-Speech Translation (fra2en) on CVSS-C](<https://www.sota2.com/research/sota/speech-to-speech-translation-fra2en-on-cvss-c.md>): Speech-to-Speech Translation (fra2en); CVSS-C; BLEU: 0.2178.

- [Speech-to-Speech Translation (fra2en) on CVSS-T](<https://www.sota2.com/research/sota/speech-to-speech-translation-fra2en-on-cvss-t.md>): Speech-to-Speech Translation (fra2en); CVSS-T; BLEU Score: 19.49.

- [Speech-to-Text Translation (en2fra) on CVSS-T](<https://www.sota2.com/research/sota/speech-to-text-translation-en2fra-on-cvss-t.md>): Speech-to-Text Translation (en2fra); CVSS-T; BLEU: 26.13.

- [Speech-to-Text Translation on CVSS-C (fra2en)](<https://www.sota2.com/research/sota/speech-to-text-translation-on-cvss-c-fra2en.md>): Speech-to-Text Translation (fra2en); CVSS-C; BLEU: 23.04.

- [Speech-to-Text Translation (fra2en) on CVSS-T](<https://www.sota2.com/research/sota/speech-to-text-translation-fra2en-on-cvss-t.md>): Speech-to-Text Translation (fra2en); CVSS-T; BLEU Score: 20.94.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2601.16618>)

[PDF](<https://arxiv.org/pdf/2601.16618.pdf>)