# w2v-SELD: A Sound Event Localization and Detection Framework for Self-Supervised Spatial Audio Pre-Training

Source: [w2v-SELD: A Sound Event Localization and Detection Framework for Self-Supervised Spatial Audio Pre-Training](<https://www.sota2.com/research/paper/68fa71a71e209131cea52539>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T02:10:33.419Z

Authors: Orlem Lima dos Santos, Karen Rosero, Roberto de Alencar Lotufo

Published: 2023-12-12T00:37:24.000Z

## Abstract

Sound Event Detection and Localization (SELD) constitutes a complex task that depends on extensive multichannel audio recordings with annotated sound events and their respective locations. In this paper, we introduce a self-supervised approach for SELD adapted from the pre-training methodology of wav2vec 2.0, which learns representations directly from raw audio data, eliminating the need for supervision. By applying this approach to SELD, we can leverage a substantial amount of unlabeled 3D audio data to learn robust representations of sound events and their locations. Our method comprises two primary stages: pre-training and fine-tuning. In the pre-training phase, unlabeled 3D audio datasets are utilized to train our w2v-SELD model, capturing intricate high-level features and contextual information inherent in audio signals. Subsequently, in the fine-tuning stage, a smaller dataset with labeled SELD data fine-tunes the pre-trained model. Experimental results on benchmark datasets demonstrate the effectiveness of the proposed self-supervised approach for SELD. The model surpasses baseline systems provided with the datasets and achieves competitive performance comparable to state-of-the-art supervised methods. The code and pre-trained parameters of our w2v-SELD model are available in this repository.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Sound Event Localization and Detection on TAU Spatial DCASE Task 3 2019 (full evaluation)](<https://www.sota2.com/research/sota/sound-event-localization-and-detection-on-tau-spatial-dcase-task-3-2019-full-evaluation.md>): Sound Event Localization and Detection; TAU Spatial DCASE Task 3 2019 (full evaluation); F1 Score: 94.7.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2312.06907>)

[PDF](<https://arxiv.org/pdf/2312.06907.pdf>)