# Unimodal Aggregation for CTC-based Speech Recognition

Source: [Unimodal Aggregation for CTC-based Speech Recognition](<https://www.sota2.com/research/paper/68fa72911e209131cea57245>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T03:52:47.950Z

Authors: Ying Fang, Xiaofei Li

Published: 2023-09-15T04:34:40.000Z

## Abstract

This paper works on non-autoregressive automatic speech recognition. A unimodal aggregation (UMA) is proposed to segment and integrate the feature frames that belong to the same text token, and thus to learn better feature representations for text tokens. The frame-wise features and weights are both derived from an encoder. Then, the feature frames with unimodal weights are integrated and further processed by a decoder. Connectionist temporal classification (CTC) loss is applied for training. Compared to the regular CTC, the proposed method learns better feature representations and shortens the sequence length, resulting in lower recognition error and computational complexity. Experiments on three Mandarin datasets show that UMA demonstrates superior or comparable performance to other advanced non-autoregressive methods, such as self-conditioned CTC. Moreover, by integrating self-conditioned CTC into the proposed framework, the performance can be further noticeably improved.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Automatic Speech Recognition on HKUST](<https://www.sota2.com/research/sota/automatic-speech-recognition-on-hkust.md>): Automatic Speech Recognition; HKUST; Sub Error Rate: 13.7.

- [Automatic Speech Recognition on AISHELL-2 android](<https://www.sota2.com/research/sota/automatic-speech-recognition-on-aishell-2-android.md>): Automatic Speech Recognition; AISHELL-2 android; CER: 6.

- [Automatic Speech Recognition on AISHELL-2 mic](<https://www.sota2.com/research/sota/automatic-speech-recognition-on-aishell-2-mic.md>): Automatic Speech Recognition; AISHELL-2 mic; CER: 5.9.

- [Automatic Speech Recognition on AISHELL-2 (test\_ios)](<https://www.sota2.com/research/sota/automatic-speech-recognition-on-aishell-2-test_ios.md>): Automatic Speech Recognition; AISHELL-2 (test\_ios); CER: 5.3.

- [Automatic Speech Recognition on AISHELL-1 (dev)](<https://www.sota2.com/research/sota/automatic-speech-recognition-on-aishell-1-dev.md>): Automatic Speech Recognition; AISHELL-1 (dev); CER: 4.4.

- [Automatic Speech Recognition on AISHELL-1 (test)](<https://www.sota2.com/research/sota/automatic-speech-recognition-on-aishell-1-test.md>): Automatic Speech Recognition; AISHELL-1 (test); CER: 4.7.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2309.08150>)

[PDF](<https://arxiv.org/pdf/2309.08150.pdf>)

[Implementation](<https://github.com/Audio-WestlakeU/UMA-ASR>)