# Beyond Binary Instrument QA: Probing Instrument Grounding in Music Audio-Language Models

Source: [Beyond Binary Instrument QA: Probing Instrument Grounding in Music Audio-Language Models](<https://www.sota2.com/research/paper/6a449ef1e0224bd4f4992e13>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-07-02T05:00:05.660Z

Authors: Yujun Lee, Joonhyeok Shin, Hyoeun Kim, Kyuhong Shim

Published: 2026-06-30T08:39:56.000Z

## Abstract

Recent music audio-language models achieve high accuracy on instrument question-answering benchmarks, but it remains unclear whether this reflects robust audio grounding or benchmark-specific shortcuts. In this paper, we introduce an OpenMIC-derived diagnostic benchmark sequence for instrument grounding in music audio-language models, extending binary instrument-presence QA to genre-prior-reduced examples, confusable instrument discrimination, longer audio context, and temporal localization. Across these settings, high binary QA accuracy often fails to predict model behavior: models can exhibit option-position bias, confusable-instrument errors, and temporal response bias. These results suggest that instrument grounding should be evaluated with multi-axis diagnostic benchmarks rather than a single aggregate accuracy.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2606.31338>)

[PDF](<https://arxiv.org/pdf/2606.31338.pdf>)
