# Omnidirectional Information Gathering for Knowledge Transfer-based Audio-Visual Navigation

Source: [Omnidirectional Information Gathering for Knowledge Transfer-based Audio-Visual Navigation](<https://www.sota2.com/research/paper/68fa70601e209131cea4ab11>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T01:40:08.455Z

Authors: Jinyu Chen, Wenguan Wang, Si Liu, Hongsheng Li, Yi Yang

Published: 2023-08-20T16:03:54.000Z

## Abstract

Audio-visual navigation is an audio-targeted wayfinding task where a robot agent is entailed to travel a never-before-seen 3D environment towards the sounding source. In this article, we present ORAN, an omnidirectional audio-visual navigator based on cross-task navigation skill transfer. In particular, ORAN sharpens its two basic abilities for a such challenging task, namely wayfinding and audio-visual information gathering. First, ORAN is trained with a confidence-aware cross-task policy distillation (CCPD) strategy. CCPD transfers the fundamental, point-to-point wayfinding skill that is well trained on the large-scale PointGoal task to ORAN, so as to help ORAN to better master audio-visual navigation with far fewer training samples. To improve the efficiency of knowledge transfer and address the domain gap, CCPD is made to be adaptive to the decision confidence of the teacher policy. Second, ORAN is equipped with an omnidirectional information gathering (OIG) mechanism, i.e., gleaning visual-acoustic observations from different directions before decision-making. As a result, ORAN yields more robust navigation behaviour. Taking CCPD and OIG together, ORAN significantly outperforms previous competitors. After the model ensemble, we got 1st in Soundspaces Challenge 2022, improving SPL and SR by 53% and 35% relatively.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Audio-driven Navigation on Matterport3D (Unheard)](<https://www.sota2.com/research/sota/audio-driven-navigation-on-matterport3d-unheard.md>): Audio-driven Navigation; Matterport3D (Unheard); SPL: 50.8.

- [Audio-visual Navigation on Replica 1.0 (Unheard)](<https://www.sota2.com/research/sota/audio-visual-navigation-on-replica-1-0-unheard.md>): Audio-visual Navigation; Replica 1.0 (Unheard); SPL: 46.7.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2308.10306>)

[PDF](<https://arxiv.org/pdf/2308.10306.pdf>)