# All That Glitters Is Not Audio: Rethinking Text Priors and Audio Reliance in Audio-Language Evaluation

Source: [All That Glitters Is Not Audio: Rethinking Text Priors and Audio Reliance in Audio-Language Evaluation](<https://www.sota2.com/research/paper/69f03f7a55a3c32af89fa542>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-29T05:00:19.306Z

Authors: Leonardo Haw-Yang Foo, Chih-Kai Yang, Chen-An Li, Ke-Han Lu, Hung-yi Lee

Published: 2026-04-27T12:25:18.000Z

## Abstract

Large Audio-Language Models show consistent performance gains across speech and audio benchmarks, yet high scores may not reflect true auditory perception. If a model can answer questions without processing the acoustic signal, the benchmark fails as a measure of auditory understanding. We present a diagnostic framework using two axes: text prior, which measures answerability from text and general knowledge alone, and audio reliance, which assesses actual dependency on the acoustic signal. Evaluating eight LALMs across three benchmarks, we find that models retain 60-72% of their full audio scores even without any audio input. Moreover, among items that require audio, only 3.0-4.2% need the complete audio clip; the majority can be resolved using localized fragments. These findings challenge the assumption that benchmark performance equals robust audio understanding, and we conclude with practical guidelines for improving evaluation reliability and benchmark design.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Audio-Language Reasoning on MMAR 1.0 (test)](<https://www.sota2.com/research/sota/audio-language-reasoning-on-mmar-1-0-test.md>): Audio-Language Reasoning; MMAR 1.0 (test); Accuracy: not recorded.

- [Audio-Language Understanding on MMAU 1.0 (test)](<https://www.sota2.com/research/sota/audio-language-understanding-on-mmau-1-0-test.md>): Audio-Language Understanding; MMAU 1.0 (test); Accuracy: not recorded.

- [Audio-Language Understanding (MCQ) on MMAU-Pro 1.0 (test)](<https://www.sota2.com/research/sota/audio-language-understanding-mcq-on-mmau-pro-1-0-test.md>): Audio-Language Understanding (MCQ); MMAU-Pro 1.0 (test); Accuracy: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2604.24401>)

[PDF](<https://arxiv.org/pdf/2604.24401.pdf>)