# Audio-Language Understanding on MMAU 1.0 (test) Source: [Audio-Language Understanding on MMAU 1.0 (test)]() Collection index: [llms.txt]() Data updated: 2026-04-28T10:21:24.811Z Evaluation of Audio-Language Understanding capabilities on the test split of the MMAU 1.0 dataset, reporting Accuracy and Text-prior rate (RTP). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Audio-Language Understanding | | Dataset (including any stated split) | MMAU 1.0 (test) | | Metric | Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 27 of 27 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy](): Higher is better; unit: PERCENT. - [Text-prior Rate (RTP)](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy | Result date | Report paper | | --- | --- | --- | --- | | Qwen3-Omni (I) | 77.4 | Not recorded | 69f03f7a55a3c32af89fa542 | | Qwen3-Omni (T) | 76.2 | Not recorded | 69f03f7a55a3c32af89fa542 | | Audio-Flamingo-3 | 75 | Not recorded | 69f03f7a55a3c32af89fa542 | | Qwen2.5-Omni-7B | 74.8 | Not recorded | 69f03f7a55a3c32af89fa542 | | DeSTA-2.5 | 65.2 | Not recorded | 69f03f7a55a3c32af89fa542 | | Qwen2-Audio-7B | 63.9 | Not recorded | 69f03f7a55a3c32af89fa542 | | Audio-Flamingo-3 | 60.9 | Not recorded | 69f03f7a55a3c32af89fa542 | | Phi-4-Multimodal | 60.4 | Not recorded | 69f03f7a55a3c32af89fa542 | | Qwen3-Omni (I) | 56.6 | Not recorded | 69f03f7a55a3c32af89fa542 | | Voxtral-Mini-3B | 55.9 | Not recorded | 69f03f7a55a3c32af89fa542 | | Qwen3-Omni (T) | 55.8 | Not recorded | 69f03f7a55a3c32af89fa542 | | Qwen3-Omni (I) | 50.8 | Not recorded | 69f03f7a55a3c32af89fa542 | | Qwen2.5-Omni-7B | 48.7 | Not recorded | 69f03f7a55a3c32af89fa542 | | Audio-Flamingo-3 | 45.5 | Not recorded | 69f03f7a55a3c32af89fa542 | | Qwen2.5-Omni-7B | 45.5 | Not recorded | 69f03f7a55a3c32af89fa542 | | Voxtral-Mini-3B | 39.6 | Not recorded | 69f03f7a55a3c32af89fa542 | | Qwen3-Omni (T) | 38.6 | Not recorded | 69f03f7a55a3c32af89fa542 | | Qwen2-Audio-7B | 38.5 | Not recorded | 69f03f7a55a3c32af89fa542 | | Qwen2-Audio-7B | 38.3 | Not recorded | 69f03f7a55a3c32af89fa542 | | Phi-4-Multimodal | 29 | Not recorded | 69f03f7a55a3c32af89fa542 | | Phi-4-Multimodal | 28.9 | Not recorded | 69f03f7a55a3c32af89fa542 | | DeSTA-2.5 | 28.4 | Not recorded | 69f03f7a55a3c32af89fa542 | | DeSTA-2.5 | 28.1 | Not recorded | 69f03f7a55a3c32af89fa542 | | Chance Level | 25 | Not recorded | 69f03f7a55a3c32af89fa542 | | Chance Level | 25 | Not recorded | 69f03f7a55a3c32af89fa542 | | Chance Level | 25 | Not recorded | 69f03f7a55a3c32af89fa542 | | Voxtral-Mini-3B | 23 | Not recorded | 69f03f7a55a3c32af89fa542 | ### Qwen3-Omni (I) Accuracy: 77.4. Result ID: 69f089b247f34ce924d71f5e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"30B†","Evaluation Protocol":"Full"}. This record does not assert independent reproduction. ### Qwen3-Omni (T) Accuracy: 76.2. Result ID: 69f089b247f34ce924d71f61. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"30B†","Evaluation Protocol":"Full"}. This record does not assert independent reproduction. ### Audio-Flamingo-3 Accuracy: 75. Result ID: 69f089b247f34ce924d71f4f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"8.4B","Evaluation Protocol":"Full"}. This record does not assert independent reproduction. ### Qwen2.5-Omni-7B Accuracy: 74.8. Result ID: 69f089b247f34ce924d71f5b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"10.7B","Evaluation Protocol":"Full"}. This record does not assert independent reproduction. ### DeSTA-2.5 Accuracy: 65.2. Result ID: 69f089b247f34ce924d71f52. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"8.8B","Evaluation Protocol":"Full"}. This record does not assert independent reproduction. ### Qwen2-Audio-7B Accuracy: 63.9. Result ID: 69f089b247f34ce924d71f58. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"8.2B","Evaluation Protocol":"Full"}. This record does not assert independent reproduction. ### Audio-Flamingo-3 Accuracy: 60.9. Result ID: 69f089b247f34ce924d71f50. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"8.4B","Evaluation Protocol":"None"}. This record does not assert independent reproduction. ### Phi-4-Multimodal Accuracy: 60.4. Result ID: 69f089b247f34ce924d71f55. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"5.6B","Evaluation Protocol":"Full"}. This record does not assert independent reproduction. ### Qwen3-Omni (I) Accuracy: 56.6. Result ID: 69f089b247f34ce924d71f5f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"30B†","Evaluation Protocol":"None"}. This record does not assert independent reproduction. ### Voxtral-Mini-3B Accuracy: 55.9. Result ID: 69f089b247f34ce924d71f64. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"4.7B","Evaluation Protocol":"Full"}. This record does not assert independent reproduction. ### Qwen3-Omni (T) Accuracy: 55.8. Result ID: 69f089b247f34ce924d71f62. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"30B†","Evaluation Protocol":"None"}. This record does not assert independent reproduction. ### Qwen3-Omni (I) Accuracy: 50.8. Result ID: 69f089b247f34ce924d71f60. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"30B†","Evaluation Protocol":"TB"}. This record does not assert independent reproduction. ### Qwen2.5-Omni-7B Accuracy: 48.7. Result ID: 69f089b247f34ce924d71f5c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"10.7B","Evaluation Protocol":"None"}. This record does not assert independent reproduction. ### Audio-Flamingo-3 Accuracy: 45.5. Result ID: 69f089b247f34ce924d71f51. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"8.4B","Evaluation Protocol":"TB"}. This record does not assert independent reproduction. ### Qwen2.5-Omni-7B Accuracy: 45.5. Result ID: 69f089b247f34ce924d71f5d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"10.7B","Evaluation Protocol":"TB"}. This record does not assert independent reproduction. ### Voxtral-Mini-3B Accuracy: 39.6. Result ID: 69f089b247f34ce924d71f65. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"4.7B","Evaluation Protocol":"None"}. This record does not assert independent reproduction. ### Qwen3-Omni (T) Accuracy: 38.6. Result ID: 69f089b247f34ce924d71f63. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"30B†","Evaluation Protocol":"TB"}. This record does not assert independent reproduction. ### Qwen2-Audio-7B Accuracy: 38.5. Result ID: 69f089b247f34ce924d71f5a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"8.2B","Evaluation Protocol":"TB"}. This record does not assert independent reproduction. ### Qwen2-Audio-7B Accuracy: 38.3. Result ID: 69f089b247f34ce924d71f59. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"8.2B","Evaluation Protocol":"None"}. This record does not assert independent reproduction. ### Phi-4-Multimodal Accuracy: 29. Result ID: 69f089b247f34ce924d71f56. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"5.6B","Evaluation Protocol":"None"}. This record does not assert independent reproduction. ### Phi-4-Multimodal Accuracy: 28.9. Result ID: 69f089b247f34ce924d71f57. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"5.6B","Evaluation Protocol":"TB"}. This record does not assert independent reproduction. ### DeSTA-2.5 Accuracy: 28.4. Result ID: 69f089b247f34ce924d71f54. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"8.8B","Evaluation Protocol":"TB"}. This record does not assert independent reproduction. ### DeSTA-2.5 Accuracy: 28.1. Result ID: 69f089b247f34ce924d71f53. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"8.8B","Evaluation Protocol":"None"}. This record does not assert independent reproduction. ### Chance Level Accuracy: 25. Result ID: 69f089b247f34ce924d71f67. [Reporting paper]() Recorded evaluation configuration: {"Size":null,"Evaluation Protocol":"Full"}. This record does not assert independent reproduction. ### Chance Level Accuracy: 25. Result ID: 69f089b247f34ce924d71f68. [Reporting paper]() Recorded evaluation configuration: {"Size":null,"Evaluation Protocol":"None"}. This record does not assert independent reproduction. ### Chance Level Accuracy: 25. Result ID: 69f089b247f34ce924d71f69. [Reporting paper]() Recorded evaluation configuration: {"Size":null,"Evaluation Protocol":"TB"}. This record does not assert independent reproduction. ### Voxtral-Mini-3B Accuracy: 23. Result ID: 69f089b247f34ce924d71f66. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Size":"4.7B","Evaluation Protocol":"TB"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.