# Massive Multimodal Audio Reasoning on MMAR Source: [Massive Multimodal Audio Reasoning on MMAR]() Collection index: [llms.txt]() Data updated: 2026-05-21T10:14:39.122Z Evaluation of Massive Multimodal Audio Reasoning capabilities on the MMAR dataset, reporting the Normalized Judge Score. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Massive Multimodal Audio Reasoning | | Dataset (including any stated split) | MMAR | | Metric | Normalized Judge Score | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 9 of 9 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Normalized Judge Score](): Higher is better; unit: SCALAR. ## Recorded results | Method | Normalized Judge Score | Result date | Report paper | | --- | --- | --- | --- | | Step-Audio-R1 | 56.4 | Not recorded | 68fa7b251e209131cea86d9e | | AF3 | 50.5 | Not recorded | 68fa7b251e209131cea86d9e | | VIRBA | 49.8 | Not recorded | 68fa7b251e209131cea86d9e | | Kimi-Audio | 45.2 | Not recorded | 68fa7b251e209131cea86d9e | | VIRBA | 42.7 | Not recorded | 68fa7b251e209131cea86d9e | | Qwen2.5-Omni | 38.9 | Not recorded | 68fa7b251e209131cea86d9e | | Single-view RL | 36 | Not recorded | 68fa7b251e209131cea86d9e | | GLM-4-Voice | 34.8 | Not recorded | 68fa7b251e209131cea86d9e | | Qwen2-Audio base | 31.4 | Not recorded | 68fa7b251e209131cea86d9e | ### Step-Audio-R1 Normalized Judge Score: 56.4. Result ID: 6a0edadbfbc373592c06a5b4. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### AF3 Normalized Judge Score: 50.5. Result ID: 6a0edadbfbc373592c06a5b3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Full name":"Audio Flamingo 3"}. This record does not assert independent reproduction. ### VIRBA Normalized Judge Score: 49.8. Result ID: 6a0edadbfbc373592c06a5b7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-Omni"}. This record does not assert independent reproduction. ### Kimi-Audio Normalized Judge Score: 45.2. Result ID: 6a0edadbfbc373592c06a5b2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### VIRBA Normalized Judge Score: 42.7. Result ID: 6a0edadbfbc373592c06a5b6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2-Audio"}. This record does not assert independent reproduction. ### Qwen2.5-Omni Normalized Judge Score: 38.9. Result ID: 6a0edadbfbc373592c06a5b1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Single-view RL Normalized Judge Score: 36. Result ID: 6a0edadbfbc373592c06a5b5. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### GLM-4-Voice Normalized Judge Score: 34.8. Result ID: 6a0edadbfbc373592c06a5b0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### Qwen2-Audio base Normalized Judge Score: 31.4. Result ID: 6a0edadbfbc373592c06a5af. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.