Audio Understanding on MMAU (test)
76.58Speech ScoreGemini 2.5 Flash
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Gemini 2.5 Flash2025.12 | 76.58 | 73.27 | 65.57 | 71.8 | — | |
| Gemini 2.0 Flash2026.01 | 72.87 | 68.93 | 59.3 | 67.03 | — | |
| Qwen2.5-Omni2025.12 | 70.6 | 78.1 | 65.9 | 71.5 | — | |
| DIFFA-2Size=8B2026.01 | 70.18 | 70.83 | 60.1 | 67 | — | |
| DIFFA-2Size=8B, factor-based parallel decoding=true2026.01 | 70.15 | 69.43 | 59.57 | 66.34 | — | |
| SpectCount (Audio Flamingo 3)Size=8.3B, Trained Parameters=26.2M, Setting=SpectCount2026.06 | 69.5 | 78.2 | 73.67 | 73.79 | — | |
| GPT-4o-Audio2026.01 | 69.33 | 63.2 | 49.93 | 60.82 | — | |
| MiMo-Audio-7B-Instruct2025.12 | 68.47 | 82.58 | 73.65 | 74.9 | — | |
| Step-Audio2-mini2025.12 | 68.16 | 79.3 | 68.44 | 72.73 | — | |
| Audio Flamingo 3Size=8.3B, Setting=Base (reproduced)2026.06 | 68.03 | 77.5 | 71.53 | 72.36 | — | |
| Qwen2.5-OmniSize=7B2026.01 | 67.93 | 69.53 | 62.5 | 66.64 | — | |
| Audio Flamingo 3Size=8.3B, Setting=Base (reported)2026.06 | 66.97 | 75.83 | 74.47 | 72.42 | — | |
| Qwen3-OmniSize=30B-A3B2026.01 | 66.46 | 73.7 | 72.22 | 70.77 | — | |
| Audio Flamingo 32025.12 | 66.37 | 79.58 | 66.77 | 73.3 | — | |
| Phi-4-multimodalSize=8B2026.01 | 63.8 | 62.67 | 61.97 | 62.81 | — | |
| Kimi-Audio-Instruct2025.12 | 62.16 | 75.68 | 66.77 | 68.2 | — | |
| Qwen2-Audio-7B-Instruct (Base Model)Backbone=Qwen2-Audio-7B-Instruct2026.01 | 58.73 | 64.74 | 58.98 | 60.81 | — | |
| SpectCount (Qwen2-Audio-Instruct)Size=8.4B, Trained Parameters=25.9M, Setting=SpectCount2026.06 | 58.6 | 69.13 | 56.13 | 61.29 | — | |
| Kimi-AudioSize=7B2026.01 | 56.57 | 70.7 | 65.93 | 64.4 | — | |
| CORDBackbone=Qwen2-Audio-7B-Instruct2026.01 | 55.42 | 64.44 | 60.18 | 60.01 | — | |
| Qwen2-AudioSize=8B2026.01 | 55.37 | 61.17 | 55.67 | 57.4 | — | |
| Qwen2-Audio-InstructSize=8.4B, Setting=Base (reported)2026.06 | 55.37 | 61.17 | 55.67 | 57.4 | — | |
| Qwen2-Audio-InstructSize=8.4B, Setting=Base (reproduced)2026.06 | 53.17 | 63.37 | 52.8 | 56.44 | — | |
| Qwen2-Audio-7B-Instruct + Forward KLBackbone=Qwen2-Audio-7B-Instruct, Training=Forward KL2026.01 | 53.01 | 61.7 | 55.99 | 56.9 | — | |
| Qwen2-Audio-7B-Instruct + SFTBackbone=Qwen2-Audio-7B-Instruct, Training=SFT2026.01 | 51.51 | 64.44 | 56.29 | 57.39 | — | |
| Baichuan-AudioSize=11B2026.01 | 42.47 | 59.46 | 49.1 | 50.34 | — | |
| GLM-4-Voice2025.12 | 35.44 | 27.63 | 27.84 | 30.3 | — | |
| GLM-4-VoiceSize=9B2026.01 | 35.44 | 27.63 | 27.84 | 30.3 | — | |
| SalmonnSize=7B2026.01 | 28.77 | 42.1 | 37.83 | 36.23 | — | |
| Qwen-Audio-ChatSize=8B2026.01 | 27.95 | 56.73 | 40.9 | 41.86 | — | |
| LTUSize=7B2026.01 | 15.33 | 20.67 | 15.68 | 17.23 | — | |
| Audio Flamingo 2Model Category=Open-source Models2026.04 | — | — | — | — | 61.06 | |
| Audio Flamingo 3Model Category=Open-source Models2026.04 | — | — | — | — | 72.42 | |
| Audio-DeepThinkerModel Category=Open-source Models Finetuned with RL2026.04 | — | — | — | — | 75.44 | |
| Audio-FlamingoModel version=32025.12 | — | — | — | — | 73.3 | |
| Audio-ReasonerModel Category=Open-source Models Finetuned with RL2026.04 | — | — | — | — | 63.78 | |
| Audio-ThinkerModel Category=Open-source Models Finetuned with RL2026.04 | — | — | — | — | 75.39 | |
| AudioMCQModel Category=Open-source Models Finetuned with RL2026.04 | — | — | — | — | 75.6 | |
| Covo-AudioModel Category=Open-source Models2026.04 | — | — | — | — | 71.89 | |
| Fun-Audio-ChatModel size=30B-A3B2025.12 | — | — | — | — | 77.9 | |
| Fun-Audio-ChatModel size=8B2025.12 | — | — | — | — | 76.6 | |
| Gemini 2.0 FlashModel Category=Closed-source Models2026.04 | — | — | — | — | 67.03 | |
| Gemini 2.5 FlashModel Category=Closed-source Models2026.04 | — | — | — | — | 67.39 | |
| GPT-4o AudioModel Category=Closed-source Models2026.04 | — | — | — | — | 60.82 | |
| GPT-4o mini AudioModel Category=Closed-source Models2026.04 | — | — | — | — | 51.03 | |
| Ke-Omni-RModel Category=Open-source Models Finetuned with RL2026.04 | — | — | — | — | 68.71 | |
| Kimi-Audio2025.12 | — | — | — | — | 69.6 | |
| Kimi-AudioModel Category=Open-source Models2026.04 | — | — | — | — | 64.4 | |
| MiMo-Audio2025.12 | — | — | — | — | 74.9 | |
| Omni-R1Model Category=Open-source Models Finetuned with RL2026.04 | — | — | — | — | 75 | |
| Qwen-2.5-OmniModel Category=Open-source Models2026.04 | — | — | — | — | 71 | |
| Qwen2-Audio-InstructModel Category=Open-source Models2026.04 | — | — | — | — | 57.4 | |
| Qwen3-Omni-30B-A3B-InstructModel Category=Open-source Models2026.04 | — | — | — | — | 74.57 | |
| Qwen3-Omni-30B-A3B-ThinkingModel Category=Open-source Models2026.04 | — | — | — | — | 74.99 | |
| Step-Audio 2Model Category=Open-source Models2026.04 | — | — | — | — | 73.86 | |
| Step-Audio2-Mini2025.12 | — | — | — | — | 73.2 |