# Audio-Language Reasoning on MMAR 1.0 (test) Source: [Audio-Language Reasoning on MMAR 1.0 (test)]() Collection index: [llms.txt]() Data updated: 2026-04-28T10:21:24.852Z Evaluation of Audio-Language Reasoning capabilities on the test split of the MMAR 1.0 dataset, measured by metrics such as Accuracy and Text-prior rate (RTP). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Audio-Language Reasoning | | Dataset (including any stated split) | MMAR 1.0 (test) | | Metric | Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 27 of 27 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy](): Higher is better; unit: PERCENT. - [Text-prior Rate (RTP)](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy | Result date | Report paper | | --- | --- | --- | --- | | Qwen3-Omni (T) | 70.3 | Not recorded | 69f03f7a55a3c32af89fa542 | | Qwen3-Omni (I) | 69.7 | Not recorded | 69f03f7a55a3c32af89fa542 | | Qwen2.5-Omni-7B | 63.9 | Not recorded | 69f03f7a55a3c32af89fa542 | | Audio-Flamingo-3 | 58.8 | Not recorded | 69f03f7a55a3c32af89fa542 | | Voxtral-Mini-3B | 50.9 | Not recorded | 69f03f7a55a3c32af89fa542 | | DeSTA-2.5 | 46.4 | Not recorded | 69f03f7a55a3c32af89fa542 | | Qwen2-Audio-7B | 46.3 | Not recorded | 69f03f7a55a3c32af89fa542 | | Phi-4-Multimodal | 46.1 | Not recorded | 69f03f7a55a3c32af89fa542 | | Qwen3-Omni (I) | 44.1 | Not recorded | 69f03f7a55a3c32af89fa542 | | Qwen3-Omni (T) | 41.9 | Not recorded | 69f03f7a55a3c32af89fa542 | | Qwen2.5-Omni-7B | 41.3 | Not recorded | 69f03f7a55a3c32af89fa542 | | Qwen3-Omni (I) | 37.6 | Not recorded | 69f03f7a55a3c32af89fa542 | | Audio-Flamingo-3 | 35.3 | Not recorded | 69f03f7a55a3c32af89fa542 | | Qwen2.5-Omni-7B | 35.3 | Not recorded | 69f03f7a55a3c32af89fa542 | | Voxtral-Mini-3B | 33.8 | Not recorded | 69f03f7a55a3c32af89fa542 | | Audio-Flamingo-3 | 33.1 | Not recorded | 69f03f7a55a3c32af89fa542 | | Qwen3-Omni (T) | 31.6 | Not recorded | 69f03f7a55a3c32af89fa542 | | Phi-4-Multimodal | 28.3 | Not recorded | 69f03f7a55a3c32af89fa542 | | Phi-4-Multimodal | 27.6 | Not recorded | 69f03f7a55a3c32af89fa542 | | Voxtral-Mini-3B | 26.3 | Not recorded | 69f03f7a55a3c32af89fa542 | | DeSTA-2.5 | 26.2 | Not recorded | 69f03f7a55a3c32af89fa542 | | DeSTA-2.5 | 26.1 | Not recorded | 69f03f7a55a3c32af89fa542 | | Qwen2-Audio-7B | 26 | Not recorded | 69f03f7a55a3c32af89fa542 | | Chance Level | 25 | Not recorded | 69f03f7a55a3c32af89fa542 | | Chance Level | 25 | Not recorded | 69f03f7a55a3c32af89fa542 | | Chance Level | 25 | Not recorded | 69f03f7a55a3c32af89fa542 | | Qwen2-Audio-7B | 22.5 | Not recorded | 69f03f7a55a3c32af89fa542 | ### Qwen3-Omni (T) Accuracy: 70.3. Result ID: 69f089b247f34ce924d71f7c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"30B†","Evaluation Protocol":"Full"}. This record does not assert independent reproduction. ### Qwen3-Omni (I) Accuracy: 69.7. Result ID: 69f089b247f34ce924d71f79. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"30B†","Evaluation Protocol":"Full"}. This record does not assert independent reproduction. ### Qwen2.5-Omni-7B Accuracy: 63.9. Result ID: 69f089b247f34ce924d71f76. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"10.7B","Evaluation Protocol":"Full"}. This record does not assert independent reproduction. ### Audio-Flamingo-3 Accuracy: 58.8. Result ID: 69f089b247f34ce924d71f6a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"8.4B","Evaluation Protocol":"Full"}. This record does not assert independent reproduction. ### Voxtral-Mini-3B Accuracy: 50.9. Result ID: 69f089b247f34ce924d71f7f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"4.7B","Evaluation Protocol":"Full"}. This record does not assert independent reproduction. ### DeSTA-2.5 Accuracy: 46.4. Result ID: 69f089b247f34ce924d71f6d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"8.8B","Evaluation Protocol":"Full"}. This record does not assert independent reproduction. ### Qwen2-Audio-7B Accuracy: 46.3. Result ID: 69f089b247f34ce924d71f73. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"8.2B","Evaluation Protocol":"Full"}. This record does not assert independent reproduction. ### Phi-4-Multimodal Accuracy: 46.1. Result ID: 69f089b247f34ce924d71f70. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"5.6B","Evaluation Protocol":"Full"}. This record does not assert independent reproduction. ### Qwen3-Omni (I) Accuracy: 44.1. Result ID: 69f089b247f34ce924d71f7a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"30B†","Evaluation Protocol":"None"}. This record does not assert independent reproduction. ### Qwen3-Omni (T) Accuracy: 41.9. Result ID: 69f089b247f34ce924d71f7d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"30B†","Evaluation Protocol":"None"}. This record does not assert independent reproduction. ### Qwen2.5-Omni-7B Accuracy: 41.3. Result ID: 69f089b247f34ce924d71f77. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"10.7B","Evaluation Protocol":"None"}. This record does not assert independent reproduction. ### Qwen3-Omni (I) Accuracy: 37.6. Result ID: 69f089b247f34ce924d71f7b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"30B†","Evaluation Protocol":"TB"}. This record does not assert independent reproduction. ### Audio-Flamingo-3 Accuracy: 35.3. Result ID: 69f089b247f34ce924d71f6c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"8.4B","Evaluation Protocol":"TB"}. This record does not assert independent reproduction. ### Qwen2.5-Omni-7B Accuracy: 35.3. Result ID: 69f089b247f34ce924d71f78. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"10.7B","Evaluation Protocol":"TB"}. This record does not assert independent reproduction. ### Voxtral-Mini-3B Accuracy: 33.8. Result ID: 69f089b247f34ce924d71f80. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"4.7B","Evaluation Protocol":"None"}. This record does not assert independent reproduction. ### Audio-Flamingo-3 Accuracy: 33.1. Result ID: 69f089b247f34ce924d71f6b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"8.4B","Evaluation Protocol":"None"}. This record does not assert independent reproduction. ### Qwen3-Omni (T) Accuracy: 31.6. Result ID: 69f089b247f34ce924d71f7e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"30B†","Evaluation Protocol":"TB"}. This record does not assert independent reproduction. ### Phi-4-Multimodal Accuracy: 28.3. Result ID: 69f089b247f34ce924d71f72. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"5.6B","Evaluation Protocol":"TB"}. This record does not assert independent reproduction. ### Phi-4-Multimodal Accuracy: 27.6. Result ID: 69f089b247f34ce924d71f71. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"5.6B","Evaluation Protocol":"None"}. This record does not assert independent reproduction. ### Voxtral-Mini-3B Accuracy: 26.3. Result ID: 69f089b247f34ce924d71f81. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"4.7B","Evaluation Protocol":"TB"}. This record does not assert independent reproduction. ### DeSTA-2.5 Accuracy: 26.2. Result ID: 69f089b247f34ce924d71f6f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"8.8B","Evaluation Protocol":"TB"}. This record does not assert independent reproduction. ### DeSTA-2.5 Accuracy: 26.1. Result ID: 69f089b247f34ce924d71f6e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"8.8B","Evaluation Protocol":"None"}. This record does not assert independent reproduction. ### Qwen2-Audio-7B Accuracy: 26. Result ID: 69f089b247f34ce924d71f74. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"8.2B","Evaluation Protocol":"None"}. This record does not assert independent reproduction. ### Chance Level Accuracy: 25. Result ID: 69f089b247f34ce924d71f82. [Reporting paper]() Recorded evaluation configuration: {"Size":null,"Evaluation Protocol":"Full"}. This record does not assert independent reproduction. ### Chance Level Accuracy: 25. Result ID: 69f089b247f34ce924d71f83. [Reporting paper]() Recorded evaluation configuration: {"Size":null,"Evaluation Protocol":"None"}. This record does not assert independent reproduction. ### Chance Level Accuracy: 25. Result ID: 69f089b247f34ce924d71f84. [Reporting paper]() Recorded evaluation configuration: {"Size":null,"Evaluation Protocol":"TB"}. This record does not assert independent reproduction. ### Qwen2-Audio-7B Accuracy: 22.5. Result ID: 69f089b247f34ce924d71f75. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"8.2B","Evaluation Protocol":"TB"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.