Audio-Language Understanding on MMAU 1.0 (test)
77.4AccuracyQwen3-Omni (I)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Qwen3-Omni (I)Size=30B†, Evaluation Protocol=Full2026.04 | 77.4 | 73.1 | |
| Qwen3-Omni (T)Size=30B†, Evaluation Protocol=Full2026.04 | 76.2 | 73.2 | |
| Audio-Flamingo-3Size=8.4B, Evaluation Protocol=Full2026.04 | 75 | 81.2 | |
| Qwen2.5-Omni-7BSize=10.7B, Evaluation Protocol=Full2026.04 | 74.8 | 65.1 | |
| DeSTA-2.5Size=8.8B, Evaluation Protocol=Full2026.04 | 65.2 | 43.1 | |
| Qwen2-Audio-7BSize=8.2B, Evaluation Protocol=Full2026.04 | 63.9 | 59.9 | |
| Audio-Flamingo-3Size=8.4B, Evaluation Protocol=None2026.04 | 60.9 | — | |
| Phi-4-MultimodalSize=5.6B, Evaluation Protocol=Full2026.04 | 60.4 | 48 | |
| Qwen3-Omni (I)Size=30B†, Evaluation Protocol=None2026.04 | 56.6 | — | |
| Voxtral-Mini-3BSize=4.7B, Evaluation Protocol=Full2026.04 | 55.9 | 70.8 | |
| Qwen3-Omni (T)Size=30B†, Evaluation Protocol=None2026.04 | 55.8 | — | |
| Qwen3-Omni (I)Size=30B†, Evaluation Protocol=TB2026.04 | 50.8 | — | |
| Qwen2.5-Omni-7BSize=10.7B, Evaluation Protocol=None2026.04 | 48.7 | — | |
| Audio-Flamingo-3Size=8.4B, Evaluation Protocol=TB2026.04 | 45.5 | — | |
| Qwen2.5-Omni-7BSize=10.7B, Evaluation Protocol=TB2026.04 | 45.5 | — | |
| Voxtral-Mini-3BSize=4.7B, Evaluation Protocol=None2026.04 | 39.6 | — | |
| Qwen3-Omni (T)Size=30B†, Evaluation Protocol=TB2026.04 | 38.6 | — | |
| Qwen2-Audio-7BSize=8.2B, Evaluation Protocol=TB2026.04 | 38.5 | — | |
| Qwen2-Audio-7BSize=8.2B, Evaluation Protocol=None2026.04 | 38.3 | — | |
| Phi-4-MultimodalSize=5.6B, Evaluation Protocol=None2026.04 | 29 | — | |
| Phi-4-MultimodalSize=5.6B, Evaluation Protocol=TB2026.04 | 28.9 | — | |
| DeSTA-2.5Size=8.8B, Evaluation Protocol=TB2026.04 | 28.4 | — | |
| DeSTA-2.5Size=8.8B, Evaluation Protocol=None2026.04 | 28.1 | — | |
| Chance LevelEvaluation Protocol=Full2026.04 | 25 | — | |
| Chance LevelEvaluation Protocol=None2026.04 | 25 | — | |
| Chance LevelEvaluation Protocol=TB2026.04 | 25 | — | |
| Voxtral-Mini-3BSize=4.7B, Evaluation Protocol=TB2026.04 | 23 | — |