# Direct Speech Translation for Automatic Subtitling

Source: [Direct Speech Translation for Automatic Subtitling](<https://www.sota2.com/research/paper/68fa70271e209131cea496de>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T01:35:15.802Z

Authors: Sara Papi, Marco Gaido, Alina Karakanta, Mauro Cettolo, Matteo Negri, Marco Turchi

Published: 2022-09-27T06:47:42.000Z

## Abstract

Automatic subtitling is the task of automatically translating the speech of audiovisual content into short pieces of timed text, i.e. subtitles and their corresponding timestamps. The generated subtitles need to conform to space and time requirements, while being synchronised with the speech and segmented in a way that facilitates comprehension. Given its considerable complexity, the task has so far been addressed through a pipeline of components that separately deal with transcribing, translating, and segmenting text into subtitles, as well as predicting timestamps. In this paper, we propose the first direct ST model for automatic subtitling that generates subtitles in the target language along with their timestamps with a single model. Our experiments on 7 language pairs show that our approach outperforms a cascade system in the same data condition, also being competitive with production tools on both in-domain and newly-released out-domain benchmarks covering new scenarios.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Automatic Subtitling on EC Short Clips (ECSC) en-es (test)](<https://www.sota2.com/research/sota/automatic-subtitling-on-ec-short-clips-ecsc-en-es-test.md>): Automatic Subtitling; EC Short Clips (ECSC) en-es (test); SubER (cased): 52.7.

- [Automatic Subtitling on EP Interviews (EPI) en-de (test)](<https://www.sota2.com/research/sota/automatic-subtitling-on-ep-interviews-epi-en-de-test.md>): Automatic Subtitling; EP Interviews (EPI) en-de (test); SubER (cased): 80.3.

- [Automatic Subtitling on EP Interviews (EPI) en-es (test)](<https://www.sota2.com/research/sota/automatic-subtitling-on-ep-interviews-epi-en-es-test.md>): Automatic Subtitling; EP Interviews (EPI) en-es (test); SubER (cased): 72.3.

- [Automatic Subtitling on MuST-Cinema (MSTCIN) en-es (test)](<https://www.sota2.com/research/sota/automatic-subtitling-on-must-cinema-mstcin-en-es-test.md>): Automatic Subtitling; MuST-Cinema (MSTCIN) en-es (test); SubER (cased): 46.8.

- [Automatic Subtitling on EC Short Clips (ECSC) en-de (test)](<https://www.sota2.com/research/sota/automatic-subtitling-on-ec-short-clips-ecsc-en-de-test.md>): Automatic Subtitling; EC Short Clips (ECSC) en-de (test); SubER (cased): 59.

- [Automatic Subtitling on MuST-Cinema (MSTCIN) en-de (test)](<https://www.sota2.com/research/sota/automatic-subtitling-on-must-cinema-mstcin-en-de-test.md>): Automatic Subtitling; MuST-Cinema (MSTCIN) en-de (test); SubER (cased): 59.9.

- [Automatic Subtitling on Overall Average (MSTCIN, ECSC, EPI) (test)](<https://www.sota2.com/research/sota/automatic-subtitling-on-overall-average-mstcin-ecsc-epi-test.md>): Automatic Subtitling; Overall Average across MSTCIN, ECSC, and EPI (test); Subtitling Error Rate (AVG): 61.4.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2209.13192>)

[PDF](<https://arxiv.org/pdf/2209.13192.pdf>)