# Audio Mamba: Selective State Spaces for Self-Supervised Audio Representations

Source: [Audio Mamba: Selective State Spaces for Self-Supervised Audio Representations](<https://www.sota2.com/research/paper/68fa73861e209131cea5cf54>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T02:51:14.866Z

Authors: Sarthak Yadav, Zheng-Hua Tan

Published: 2024-06-04T10:19:14.000Z

## Abstract

Despite its widespread adoption as the prominent neural architecture, the Transformer has spurred several independent lines of work to address its limitations. One such approach is selective state space models, which have demonstrated promising results for language modelling. However, their feasibility for learning self-supervised, general-purpose audio representations is yet to be investigated. This work proposes Audio Mamba, a selective state space model for learning general-purpose audio representations from randomly masked spectrogram patches through self-supervision. Empirical results on ten diverse audio recognition downstream tasks show that the proposed models, pretrained on the AudioSet dataset, consistently outperform comparable self-supervised audio spectrogram transformer (SSAST) baselines by a considerable margin and demonstrate better performance in dataset size, sequence length and model size comparisons.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Synthetic speech attribution and open-set detection on DiffSSD (dev)](<https://www.sota2.com/research/sota/synthetic-speech-attribution-and-open-set-detection-on-diffssd-dev.md>): Synthetic speech attribution and open-set detection; DiffSSD (dev); ACC: 0.9641.

- [Synthetic speech attribution and open-set detection on DiffSSD (test)](<https://www.sota2.com/research/sota/synthetic-speech-attribution-and-open-set-detection-on-diffssd-test.md>): Synthetic speech attribution and open-set detection; DiffSSD (test); Accuracy: 93.4.

- [Synthetic speech attribution and open-set detection on DiffSSD (ID/OOD)](<https://www.sota2.com/research/sota/synthetic-speech-attribution-and-open-set-detection-on-diffssd-id-ood.md>): Synthetic speech attribution and open-set detection; DiffSSD (ID/OOD); Accuracy: 72.75.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2406.02178>)

[PDF](<https://arxiv.org/pdf/2406.02178.pdf>)