# Real-Time Target Sound Extraction

Source: [Real-Time Target Sound Extraction](<https://www.sota2.com/research/paper/68fa6f0f1e209131cea439c4>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-01-04T03:41:15.412Z

Authors: Bandhav Veluri, Justin Chan, Malek Itani, Tuochao Chen, Takuya Yoshioka, Shyamnath Gollakota

Published: 2022-11-04T03:51:23.000Z

## Abstract

We present the first neural network model to achieve real-time and streaming target sound extraction. To accomplish this, we propose Waveformer, an encoder-decoder architecture with a stack of dilated causal convolution layers as the encoder, and a transformer decoder layer as the decoder. This hybrid architecture uses dilated causal convolutions for processing large receptive fields in a computationally efficient manner while also leveraging the generalization performance of transformer-based architectures. Our evaluations show as much as 2.2-3.3 dB improvement in SI-SNRi compared to the prior models for this task while having a 1.2-4x smaller model size and a 1.5-2x lower runtime. We provide code, dataset, and audio samples: https://waveformer.cs.washington.edu/.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Multi-target Sound Extraction on FSD Kaggle + TAU Urban Acoustic Scenes synthetic mixture 2018 2019 (test)](<https://www.sota2.com/research/sota/multi-target-sound-extraction-on-fsd-kaggle-plus-tau-urban-acoustic-scenes-synthetic-mixture-2018-2019-test.md>): Multi-target Sound Extraction; FSD Kaggle + TAU Urban Acoustic Scenes synthetic mixture 2018 2019 (test); SI-SNRi (1 Class): 9.39.

- [Single-target Sound Extraction on FSD Kaggle 2018 and TAU Urban Acoustic Scenes 2019 (test)](<https://www.sota2.com/research/sota/single-target-sound-extraction-on-fsd-kaggle-2018-and-tau-urban-acoustic-scenes-2019-test.md>): Single-target sound extraction; FSD Kaggle 2018 and TAU Urban Acoustic Scenes 2019 (test); SI-SNRi: 9.43.

- [Target Sound Extraction on testset (test)](<https://www.sota2.com/research/sota/target-sound-extraction-on-testset-test.md>): Target Sound Extraction; testset (test); SI-SNRi: 11.31.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2211.02250>)

[PDF](<https://arxiv.org/pdf/2211.02250.pdf>)

[Implementation](<https://waveformer.cs.washington.edu/>)