# SBAAM! Eliminating Transcript Dependency in Automatic Subtitling

Source: [SBAAM! Eliminating Transcript Dependency in Automatic Subtitling](<https://www.sota2.com/research/paper/68fa73491e209131cea5b8d2>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T04:49:09.354Z

Authors: Marco Gaido, Sara Papi, Matteo Negri, Mauro Cettolo, Luisa Bentivogli

Published: 2024-05-17T12:42:56.000Z

## Abstract

Subtitling plays a crucial role in enhancing the accessibility of audiovisual content and encompasses three primary subtasks: translating spoken dialogue, segmenting translations into concise textual units, and estimating timestamps that govern their on-screen duration. Past attempts to automate this process rely, to varying degrees, on automatic transcripts, employed diversely for the three subtasks. In response to the acknowledged limitations associated with this reliance on transcripts, recent research has shifted towards transcription-free solutions for translation and segmentation, leaving the direct generation of timestamps as uncharted territory. To fill this gap, we introduce the first direct model capable of producing automatic subtitles, entirely eliminating any dependence on intermediate transcripts also for timestamp prediction. Experimental results, backed by manual evaluation, showcase our solution's new state-of-the-art performance across multiple language pairs and diverse conditions.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Automatic Subtitling on MuST-Cinema (MSTCIN) en-de (test)](<https://www.sota2.com/research/sota/automatic-subtitling-on-must-cinema-mstcin-en-de-test.md>): Automatic Subtitling; MuST-Cinema (MSTCIN) en-de (test); SubER (cased): 56.2.

- [Automatic Subtitling on MuST-Cinema (MSTCIN) en-es (test)](<https://www.sota2.com/research/sota/automatic-subtitling-on-must-cinema-mstcin-en-es-test.md>): Automatic Subtitling; MuST-Cinema (MSTCIN) en-es (test); SubER (cased): 44.7.

- [Automatic Subtitling on EC Short Clips (ECSC) en-de (test)](<https://www.sota2.com/research/sota/automatic-subtitling-on-ec-short-clips-ecsc-en-de-test.md>): Automatic Subtitling; EC Short Clips (ECSC) en-de (test); SubER (cased): 58.5.

- [Automatic Subtitling on Overall Average (MSTCIN, ECSC, EPI) (test)](<https://www.sota2.com/research/sota/automatic-subtitling-on-overall-average-mstcin-ecsc-epi-test.md>): Automatic Subtitling; Overall Average across MSTCIN, ECSC, and EPI (test); Subtitling Error Rate (AVG): 59.2.

- [Automatic Subtitling on IWSLT en-de 2023 (val)](<https://www.sota2.com/research/sota/automatic-subtitling-on-iwslt-en-de-2023-val.md>): Automatic Subtitling; IWSLT en-de 2023 (val); SubER (TED uncased): 62.1.

- [Automatic Subtitling on IWSLT en-es 2023 (val)](<https://www.sota2.com/research/sota/automatic-subtitling-on-iwslt-en-es-2023-val.md>): Automatic Subtitling; IWSLT en-es 2023 (val); SubER (TED cased): 49.5.

- [Automatic Subtitling on EC Short Clips (ECSC) en-es (test)](<https://www.sota2.com/research/sota/automatic-subtitling-on-ec-short-clips-ecsc-en-es-test.md>): Automatic Subtitling; EC Short Clips (ECSC) en-es (test); SubER (cased): 49.9.

- [Automatic Subtitling on EP Interviews (EPI) en-de (test)](<https://www.sota2.com/research/sota/automatic-subtitling-on-ep-interviews-epi-en-de-test.md>): Automatic Subtitling; EP Interviews (EPI) en-de (test); SubER (cased): 78.5.

- [Automatic Subtitling on EP Interviews (EPI) en-es (test)](<https://www.sota2.com/research/sota/automatic-subtitling-on-ep-interviews-epi-en-es-test.md>): Automatic Subtitling; EP Interviews (EPI) en-es (test); SubER (cased): 70.2.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2405.10741>)

[PDF](<https://arxiv.org/pdf/2405.10741.pdf>)

[Implementation](<https://github.com/hlt-mt/FBK-fairseq/>)