Audio Understanding on MMAU v05.15.25 (test)
84.04Sound ScoreStep-Audio 2
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Step-Audio 2Param=32B2026.02 | 84.04 | 73.56 | 75.15 | 77.58 | |
| MiMo-AudioParam=7B2026.02 | 81.68 | 74.25 | 68.17 | 74.7 | |
| Step-Audio 2Params=32B2026.02 | 80.6 | 68.23 | 72.75 | 73.86 | |
| Audio Flamingo 3Param=7B2026.02 | 79.58 | 73.95 | 66.37 | 73.3 | |
| Step-Audio 2 miniParam=7B2026.02 | 79.3 | 68.44 | 68.16 | 72.73 | |
| Covo-AudioParam=7B2026.02 | 78.68 | 76.05 | 71.17 | 75.3 | |
| Qwen2.5-OmniParam=7B2026.02 | 78.1 | 65.9 | 70.6 | 71.5 | |
| MiMo-AudioParams=7B2026.02 | 77.2 | 69.73 | 70.77 | 72.59 | |
| Qwen2.5-OmniSize=7B, No LLM Modification=×, # Tokens=0.3T2026.03 | 76.8 | 67.3 | 68.9 | 71 | |
| Qwen2.5-OmniParams=7B2026.02 | 76.77 | 67.33 | 68.9 | 71 | |
| Audio Flamingo 3Params=7B2026.02 | 75.83 | 74.47 | 66.97 | 72.42 | |
| Audio Flamingo 3Size=8B, No LLM Modification=×, # Samples=26.7M, # Hours=54.4K2026.03 | 75.8 | 74.5 | 67 | 72.4 | |
| Kimi-AudioParam=7B2026.02 | 75.68 | 66.77 | 62.16 | 68.2 | |
| Step-Audio 2 miniParams=7B2026.02 | 75.57 | 66.85 | 66.49 | 70.23 | |
| Gemini 2.5 ProParam=-2026.02 | 75.08 | 68.26 | 71.47 | 71.6 | |
| Covo-AudioParams=7B2026.02 | 73.9 | 70.2 | 71.56 | 71.89 | |
| Gemini 2.5 FlashParam=-2026.02 | 73.27 | 65.57 | 76.58 | 71.8 | |
| Audio Flamingo 2Param=3.2B2026.02 | 71.47 | 70.96 | 44.74 | 62.4 | |
| Gemini 2.0 FlashParam=-2026.02 | 71.17 | 65.27 | 75.08 | 70.5 | |
| Kimi-AudioParams=7B2026.02 | 70.7 | 65.93 | 56.57 | 64.4 | |
| Gemini 2.5 Pro2026.02 | 70.63 | 64.77 | 72.67 | 69.36 | |
| Gemini 2.5 Flash2026.02 | 69.5 | 69.4 | 68.27 | 67.39 | |
| Gemini 2.0 Flash2026.02 | 68.93 | 59.3 | 72.87 | 67.03 | |
| Audio Flamingo 2Params=3.2B2026.02 | 68.13 | 70.2 | 44.87 | 61.06 | |
| Audio Flamingo 2Size=3B, No LLM Modification=×, # Samples=5.9M2026.03 | 68.1 | 70.2 | 44.9 | 61.1 | |
| Audio ReasonerSize=8B, No LLM Modification=×, # Samples=> 1.2M, # Hours=> 320K2026.03 | 67.3 | 61.5 | 62.5 | 63.8 | |
| Qwen2-Audio-InstructParam=8.4B2026.02 | 67.27 | 56.29 | 55.26 | 59.6 | |
| DeSTA-2.5-AudioSize=8B, No LLM Modification=✓, # Samples=5M, # Hours=7K2026.03 | 66.9 | 57.1 | 71.9 | 65.2 | |
| GPT-4o AudioParam=-2026.02 | 64.56 | 56.29 | 66.67 | 62.5 | |
| GPT-4o Audio2026.02 | 63.2 | 49.93 | 69.33 | 60.82 | |
| GPT-4o Audio2026.03 | 63.2 | 49.9 | 69.3 | 60.8 | |
| Phi-4-MultimodalSize=4B, No LLM Modification=×, # Samples=> 28M, # Hours=> 2M2026.03 | 62.7 | 62 | 63.8 | 62.8 | |
| Qwen2-Audio-InstructSize=7B, No LLM Modification=×, # Hours=320K2026.03 | 61.2 | 55.7 | 55.4 | 57.4 | |
| Qwen2-Audio-InstructParams=8.4B2026.02 | 61.17 | 55.67 | 55.37 | 57.4 | |
| ALARM-CASize=4B, No LLM Modification=✓, # Samples=5.5M, # Hours=17K, # Tokens=1.5B2026.03 | 61.1 | 53.3 | 61.1 | 58.5 | |
| AL-SSLAM-RSize=4B, No LLM Modification=✓, # Samples=0.4M, # Hours=1.2K, # Tokens=108M2026.03 | 60.4 | 50.9 | 47.8 | 53 | |
| ALARM-ESize=4B, No LLM Modification=✓, # Samples=5.5M, # Hours=17K, # Tokens=1.5B2026.03 | 59.1 | 54.2 | 73.7 | 62.4 | |
| ALARM-PSize=4B, No LLM Modification=✓, # Samples=5.5M, # Hours=17K, # Tokens=1.5B2026.03 | 58.5 | 53.3 | 66.7 | 59.5 | |
| AL-Whisper-RSize=4B, No LLM Modification=✓, # Samples=5.5M, # Hours=17K, # Tokens=1.5B2026.03 | 55.6 | 49.7 | 58.8 | 54.7 | |
| GPT-4o mini AudioParam=-2026.02 | 50.75 | 39.22 | 69.07 | 53 | |
| Gemma 3nSize=8B4E2026.03 | 50.2 | 53.2 | 62.1 | 55.2 | |
| AL-Whisper-Instruct-RSize=4B, No LLM Modification=✓, # Samples=0.5M, # Hours=1K, # Tokens=91M2026.03 | 49.9 | 49.7 | 73 | 57.5 | |
| GPT-4o mini Audio2026.03 | 49.7 | 36 | 67.5 | 51 | |
| GPT-4o mini Audio2026.02 | 49.67 | 35.97 | 67.47 | 51.03 | |
| AL-MuQ-RSize=4B, No LLM Modification=✓, # Samples=0.4M, # Hours=1.5K, # Tokens=137M2026.03 | 49.1 | 53.3 | 43.1 | 48.5 | |
| Gemma 3nSize=6B2E2026.03 | 47.5 | 51.6 | 57 | 52 | |
| AL-W2VBERT2-RSize=4B, No LLM Modification=✓, # Samples=0.5M, # Hours=0.6K, # Tokens=58M2026.03 | 46.9 | 46 | 48.2 | 47 | |
| SALMONNSize=13B, No LLM Modification=×, # Samples=2.3M, # Hours=4.4K2026.03 | 42.1 | 37.8 | 28.8 | 36.2 | |
| MusiLingoSize=7B, No LLM Modification=✓, # Samples=0.7M2026.03 | 41.9 | 41.2 | 31.7 | 38.3 | |
| GAMA-ITSize=7B, No LLM Modification=×, # Samples=6.5M2026.03 | 32.7 | 22.4 | 11.6 | 22.2 | |
| MU-LLaMASize=7B, No LLM Modification=✓, # Samples=0.1M, # Hours=762026.03 | 31 | 29.7 | 17.1 | 25.9 | |
| GAMASize=7B, No LLM Modification=×, # Samples=6.3M2026.03 | 30.7 | 17.3 | 16.7 | 21.7 | |
| LTUSize=7B, No LLM Modification=×, # Samples=5.6M2026.03 | 20.7 | 15.7 | 15.3 | 17.2 |