Audio Understanding on MMAU
81.1AccuracyGemini-3.1-Pro
Evaluation results
| Method | Links | |
|---|---|---|
| Gemini-3.1-ProSize=–, Model Category=Closed-source models2026.06 | 81.1 | |
| Gemini-3-Pro2026.02 | 80.8 | |
| ERNIE 5.0-Base2026.02 | 80.8 | |
| ERNIE 5.02026.02 | 80.4 | |
| Gemini-3-ProSize=–, Model Category=Closed-source models2026.06 | 80.15 | |
| Step-Audio-R1Size=33B, Model Category=Open-source models2026.06 | 78.67 | |
| Qwen3-Omni-A3B-InstructVariant=Omni-A3B-Instruct2026.03 | 78.2 | |
| MOSS-Audio-4B-ThinkingSize=4B, Model Category=Open-source models2026.06 | 77.64 | |
| BaseBase Model=Qwen3-Omni-30B-A3B-Instruct2026.05 | 77.6 | |
| Qwen3-Omni-Instruct2026.02 | 77.5 | |
| MOSS-Audio-8B-ThinkingSize=8B, Model Category=Open-source models2026.06 | 77.33 | |
| MOSS-Audio-8B-InstructSize=8B, Model Category=Open-source models2026.06 | 77.03 | |
| ReGapBase Model=MiniCPM-o-4.52026.05 | 76.8 | |
| ReGapBase Model=Qwen3-Omni-30B-A3B-Instruct2026.05 | 76.8 | |
| LongCat-Next2026.03 | 76.4 | |
| BaseBase Model=MiniCPM-o-4.52026.05 | 76.2 | |
| Audio Flamingo NextSize=8B, Model Category=Open-source models2026.06 | 76.1 | |
| LongCat-Flash-Omni2026.02 | 75.9 | |
| MiMo-Audio2026.03 | 75.8 | |
| MOSS-Audio-4B-InstructSize=4B, Model Category=Open-source models2026.06 | 75.79 | |
| Qwen3-Omni-30B-A3B-InstructSize=30B, Model Category=Open-source models2026.06 | 75 | |
| MiMo-Audio-7BSize=7B, Model Category=Open-source models2026.06 | 74.9 | |
| Gemini-2.5-Flash-Lite-previewVersion=2.5, Tier=Flash-Lite-preview2026.03 | 74.8 | |
| MiMo-Audio + SICL-AT3few-shot=true2026.01 | 73.4 | |
| MiMo-Audio + SICL-AT2few-shot=true2026.01 | 72.9 | |
| Qwen2.5-Omni-3BModel Scale=3B2026.06 | 72.8 | |
| MiMo-Audio + Vanilla SICLfew-shot=true2026.01 | 72.6 | |
| Kimi-AudioSize=7B, Model Category=Open-source models2026.06 | 72.41 | |
| Step-Audio-R1.1Size=33B, Model Category=Open-source models2026.06 | 72.18 | |
| Qwen2.5-Omni + SICL-AT3few-shot=true2026.01 | 72.1 | |
| MiMo-Audio + SICL-AT1few-shot=true2026.01 | 71.9 | |
| Gemini-3.1-Flash-Lite-previewVersion=3.1, Tier=Flash-Lite-preview2026.03 | 71.7 | |
| Step-Audio-2-miniVersion=2-mini2026.03 | 71.3 | |
| ShiftDCBase Model=Qwen3-Omni-30B-A3B-Instruct2026.05 | 71.2 | |
| Qwen2.5-Omni + SICL-AT2few-shot=true2026.01 | 71.1 | |
| MiniCPM-o-4.5Size=9B, Model Category=Open-source models2026.06 | 70.97 | |
| Kimi-Audio2026.03 | 70.31 | |
| Qwen2.5-Omni + SICL-AT1few-shot=true2026.01 | 69.6 | |
| GPT-4o-Audio2026.02 | 68.4 | |
| AVLM-2BModel Scale=2B2026.06 | 68.3 | |
| Qwen2.5-Omni + Vanilla SICLfew-shot=true2026.01 | 67.3 | |
| MiMo-Audiofew-shot=false2026.01 | 66.9 | |
| ReGapBase Model=Qwen2.5-Omni-3B2026.05 | 66.8 | |
| BaseBase Model=Qwen2.5-Omni-3B2026.05 | 66 | |
| Qwen2.5-Omnifew-shot=false2026.01 | 65.8 | |
| GPT4o-AudioSize=–, Model Category=Closed-source models2026.06 | 65.66 | |
| Qwen2.5-OmniSize=7B, Model Category=Open-source models2026.06 | 65.6 | |
| Qwen2.5-Omni (Fine-tuned on RSR)few-shot=false2026.01 | 65.3 | |
| Kimi Audio2026.02 | 65.2 | |
| SPA-VL-DPOBase Model=Qwen2.5-Omni-3B2026.05 | 64.4 | |
| Qwen2.5-Omni (Fine-tuned on CV-en) + Vanilla SICLfew-shot=true2026.01 | 64.1 | |
| VLGuard-SFTBase Model=Qwen2.5-Omni-3B2026.05 | 64 | |
| Qwen2.5-Omni (Fine-tuned on CV-en)few-shot=false2026.01 | 63.9 | |
| Audio Flamingo 3Size=7B, Model Category=Open-source models2026.06 | 61.23 | |
| Gemma-4-E2B-itModel Scale=4B2026.06 | 55.9 | |
| ShiftDCBase Model=MiniCPM-o-4.52026.05 | 29.2 | |
| ShiftDCBase Model=Qwen2.5-Omni-3B2026.05 | 3.2 |