# Audio Reasoning on MMAU v05.15.25 (base)

Source: [Audio Reasoning on MMAU v05.15.25 (base)](<https://www.sota2.com/research/sota/audio-reasoning-on-mmau-v05-15-25-base>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-06T07:18:20.704Z

Evaluation of Audio Reasoning capabilities using the base version (v05.15.25) of the MMAU dataset, focusing on Sound, Music, and Speech sub-categories.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Audio Reasoning |
| Dataset (including any stated split) | MMAU v05.15.25 (base) |
| Metric | Sound Accuracy |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 10 of 10 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Sound Accuracy](<https://www.sota2.com/research/sota/audio-reasoning-on-mmau-v05-15-25-base.md?metric=Sound%20Accuracy>): Higher is better; unit: PERCENT.

- [Music Accuracy](<https://www.sota2.com/research/sota/audio-reasoning-on-mmau-v05-15-25-base.md?metric=Music%20Accuracy>): Higher is better; unit: PERCENT.

- [Speech Accuracy](<https://www.sota2.com/research/sota/audio-reasoning-on-mmau-v05-15-25-base.md?metric=Speech%20Accuracy>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Sound Accuracy | Result date | Report paper |
| --- | --- | --- | --- |
| SAM+OR-780M | 59.73 | Not recorded | 68fa7b721e209131cea8883d |
| SAM+OR-2.7B | 56.77 | Not recorded | 68fa7b721e209131cea8883d |
| Gemma 3n-4B | 50.27 | Not recorded | 68fa7b721e209131cea8883d |
| Gemma 3n-2B | 47.47 | Not recorded | 68fa7b721e209131cea8883d |
| SAM+OR-130M | 46.93 | Not recorded | 68fa7b721e209131cea8883d |
| GAMA-7B | 30.73 | Not recorded | 68fa7b721e209131cea8883d |
| LTU-7B | 25.86 | Not recorded | 68fa7b721e209131cea8883d |
| SAM-2.7B | 22.83 | Not recorded | 68fa7b721e209131cea8883d |
| SAM-130M | 11.67 | Not recorded | 68fa7b721e209131cea8883d |
| SAM-780M | 7.8 | Not recorded | 68fa7b721e209131cea8883d |

### SAM+OR-780M

Sound Accuracy: 59.73. Result ID: 69aa7e206dbb7075b953c77e.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7b721e209131cea8883d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b721e209131cea8883d.md>)

Recorded evaluation configuration: {"Size":"780M","Supervision":"OpenReasonAQA"}. This record does not assert independent reproduction.

### SAM+OR-2.7B

Sound Accuracy: 56.77. Result ID: 69aa7e206dbb7075b953c77d.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7b721e209131cea8883d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b721e209131cea8883d.md>)

Recorded evaluation configuration: {"Size":"2.7B","Supervision":"OpenReasonAQA"}. This record does not assert independent reproduction.

### Gemma 3n-4B

Sound Accuracy: 50.27. Result ID: 69aa7e206dbb7075b953c776.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7b721e209131cea8883d.md>)

Recorded evaluation configuration: {"Size":"4B"}. This record does not assert independent reproduction.

### Gemma 3n-2B

Sound Accuracy: 47.47. Result ID: 69aa7e206dbb7075b953c777.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7b721e209131cea8883d.md>)

Recorded evaluation configuration: {"Size":"2B"}. This record does not assert independent reproduction.

### SAM+OR-130M

Sound Accuracy: 46.93. Result ID: 69aa7e206dbb7075b953c77f.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7b721e209131cea8883d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b721e209131cea8883d.md>)

Recorded evaluation configuration: {"Size":"130M","Supervision":"OpenReasonAQA"}. This record does not assert independent reproduction.

### GAMA-7B

Sound Accuracy: 30.73. Result ID: 69aa7e206dbb7075b953c778.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7b721e209131cea8883d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73a71e209131cea5db04.md>)

Recorded evaluation configuration: {"Size":"7B"}. This record does not assert independent reproduction.

### LTU-7B

Sound Accuracy: 25.86. Result ID: 69aa7e206dbb7075b953c779.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7b721e209131cea8883d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72431e209131cea555aa.md>)

[Implementation](<https://github.com/YuanGongND/ltu>)

Recorded evaluation configuration: {"Size":"7B"}. This record does not assert independent reproduction.

### SAM-2.7B

Sound Accuracy: 22.83. Result ID: 69aa7e206dbb7075b953c77a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7b721e209131cea8883d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b721e209131cea8883d.md>)

Recorded evaluation configuration: {"Size":"2.7B"}. This record does not assert independent reproduction.

### SAM-130M

Sound Accuracy: 11.67. Result ID: 69aa7e206dbb7075b953c77c.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7b721e209131cea8883d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b721e209131cea8883d.md>)

Recorded evaluation configuration: {"Size":"130M"}. This record does not assert independent reproduction.

### SAM-780M

Sound Accuracy: 7.8. Result ID: 69aa7e206dbb7075b953c77b.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7b721e209131cea8883d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b721e209131cea8883d.md>)

Recorded evaluation configuration: {"Size":"780M"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.