# MURMUR: An Efficient Inference System for Long-Form ASR

Source: [MURMUR: An Efficient Inference System for Long-Form ASR](<https://www.sota2.com/research/paper/6a1e63d3dbd7150506f85445>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-02T05:02:08.515Z

Authors: Wei-Tzu Lee, Keisuke Kamahori, Baris Kasikci

Published: 2026-05-31T22:54:57.000Z

## Abstract

Long-form automatic speech recognition (ASR) requires both high accuracy and low latency, but existing systems force a trade-off between the two. Chunk-based pipelines process audio in parallel windows for low latency, but lose cross-chunk context and need brittle heuristics to align speakers and timestamps at boundaries. Long-context ASR models resolve everything in a single pass for better accuracy, but are an order of magnitude slower. We propose Murmur, an inference system that overcomes this trade-off by operating at two levels. At the inter-chunk level, we revisit the chunk-based pipeline for modern long-context ASR, treating chunk size as a tunable hyperparameter, and show that intermediate chunk sizes strike a good balance of accuracy and latency. At the intra-chunk level, we exploit attention sparsity through a sliding window KV cache eviction policy applied to both output and speech tokens. On AMI-IHM, Murmur matches single-pass accuracy while reducing latency by 4.2x, with further gains from token eviction at less than 1% relative tcpWER degradation. The code of Murmur is available at https://github.com/uw-syfi/Murmur.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Speech Recognition and Diarization on AMI IHM](<https://www.sota2.com/research/sota/speech-recognition-and-diarization-on-ami-ihm.md>): Speech Recognition and Diarization; AMI IHM; WER: 19.8.

- [Automatic Speech Recognition on AMI IHM](<https://www.sota2.com/research/sota/automatic-speech-recognition-on-ami-ihm.md>): Automatic Speech Recognition; AMI IHM; WER: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2606.01483>)

[PDF](<https://arxiv.org/pdf/2606.01483.pdf>)