# Segmental Attention Decoding With Long Form Acoustic Encodings

Source: [Segmental Attention Decoding With Long Form Acoustic Encodings](<https://www.sota2.com/research/paper/694239304561469362850761>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-12-18T05:00:11.399Z

Authors: Pawel Swietojanski, Xinwei Li, Mingbin Xu, Takaaki Hori, Dogan Can, Xiaodan Zhuang

Published: 2025-12-16T18:12:37.000Z

## Abstract

We address the fundamental incompatibility of attention-based encoder-decoder (AED) models with long-form acoustic encodings. AED models trained on segmented utterances learn to encode absolute frame positions by exploiting limited acoustic context beyond segment boundaries, but fail to generalize when decoding long-form segments where these cues vanish. The model loses ability to order acoustic encodings due to permutation invariance of keys and values in cross-attention. We propose four modifications: (1) injecting explicit absolute positional encodings into cross-attention for each decoded segment, (2) long-form training with extended acoustic context to eliminate implicit absolute position encoding, (3) segment concatenation to cover diverse segmentations needed during training, and (4) semantic segmentation to align AED-decoded segments with training segments. We show these modifications close the accuracy gap between continuous and segmented acoustic encodings, enabling auto-regressive use of the attention decoder.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Automatic Speech Recognition on TED-LIUM3 (test)](<https://www.sota2.com/research/sota/automatic-speech-recognition-on-ted-lium3-test.md>): Automatic Speech Recognition; TED-LIUM3 (test); WER: 0.039.

- [Automatic Speech Recognition on LibriSpeech clean segmented (test)](<https://www.sota2.com/research/sota/automatic-speech-recognition-on-librispeech-clean-segmented-test.md>): Automatic Speech Recognition; LibriSpeech clean segmented (test); WER: 1.7.

- [Automatic Speech Recognition on LibriSpeech other segmented (test)](<https://www.sota2.com/research/sota/automatic-speech-recognition-on-librispeech-other-segmented-test.md>): Automatic Speech Recognition; LibriSpeech other segmented (test); WER: 3.9.

- [Automatic Speech Recognition on CommonVoice segmented (test)](<https://www.sota2.com/research/sota/automatic-speech-recognition-on-commonvoice-segmented-test.md>): Automatic Speech Recognition; CommonVoice segmented (test); WER: 11.4.

- [Automatic Speech Recognition on Earnings21 long-form (test)](<https://www.sota2.com/research/sota/automatic-speech-recognition-on-earnings21-long-form-test.md>): Automatic Speech Recognition; Earnings21 long-form (test); WER: 11.4.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2512.14652>)

[PDF](<https://arxiv.org/pdf/2512.14652.pdf>)