Audio Understanding on MMAR
83.7Average ScoreGemini-3.1 Pro
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Gemini-3.1 Pro2026.04 | 83.7 | — | — | — | — | |
| Qwen3.5-Omni-Plus2026.04 | 80 | — | — | — | — | |
| Qwen3.5-Omni-Flash2026.04 | 74 | — | — | — | — | |
| Gemini-2.5-Pro2026.02 | 73.2 | 74.7 | — | — | — | |
| Qwen2.5-OmniSize=7B2026.02 | 64.1 | 56.7 | — | — | — | |
| GPT-4o Audio2026.02 | 63 | 63.5 | — | — | — | |
| OmniFysicsSize=3B2026.02 | 61.2 | 56.8 | — | — | — | |
| Qwen2.5-OmniSize=3B2026.02 | 60.1 | 53.8 | — | — | — | |
| Kimi-AudioSize=7B2026.02 | 53.5 | 38.8 | — | — | — | |
| Audio ReasonerSize=7B2026.02 | 52.3 | 36.8 | — | — | — | |
| Baichuan-Omni-1.5Size=7B2026.02 | 51.6 | 40.7 | — | — | — | |
| Gemma 3nSize=4B2026.02 | 49.1 | 40.1 | — | — | — | |
| Qwen2-Audio-InstructSize=7B2026.02 | 46.5 | 33.3 | — | — | — | |
| Gemma 3nSize=2B2026.02 | 44.8 | 37.8 | — | — | — | |
| Audio Flamingo 2Size=3B2026.02 | 42.2 | 21.9 | — | — | — | |
| AF-Next-Captioner2026.04 | — | — | — | — | 63 | |
| AF-Next-Instruct2026.04 | — | — | — | — | 59.7 | |
| AF-Next-Think2026.04 | — | — | — | — | 61 | |
| Audio Flamingo 32026.04 | — | — | — | — | 58.5 | |
| Audio-Flamingo-3Setting=Zero-shot2026.03 | — | — | 59.23 | 61.64 | — | |
| Audio-ReasonerSetting=Zero-shot2026.03 | — | — | 44.34 | 38.23 | — | |
| EvASetting=Zero-shot2026.03 | — | — | 59.79 | 59.45 | — | |
| Kimi-Audio-7B-InstructSetting=Zero-shot2026.03 | — | — | 56.79 | 58.72 | — | |
| Qwen2.5-Omni-7BSetting=Zero-shot2026.03 | — | — | 48.68 | 59.2 | — | |
| R1-AQASetting=Zero-shot2026.03 | — | — | 48.75 | 50.19 | — | |
| Step-Audio-2-miniSetting=Zero-shot2026.03 | — | — | 54.58 | 61.01 | — |