Audio-Language Reasoning on MMAR 1.0 (test)
70.3AccuracyQwen3-Omni (T)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Qwen3-Omni (T)Size=30B†, Evaluation Protocol=Full2026.04 | 70.3 | 59.6 | |
| Qwen3-Omni (I)Size=30B†, Evaluation Protocol=Full2026.04 | 69.7 | 63.3 | |
| Qwen2.5-Omni-7BSize=10.7B, Evaluation Protocol=Full2026.04 | 63.9 | 64.6 | |
| Audio-Flamingo-3Size=8.4B, Evaluation Protocol=Full2026.04 | 58.8 | 56.3 | |
| Voxtral-Mini-3BSize=4.7B, Evaluation Protocol=Full2026.04 | 50.9 | 66.4 | |
| DeSTA-2.5Size=8.8B, Evaluation Protocol=Full2026.04 | 46.4 | 56.2 | |
| Qwen2-Audio-7BSize=8.2B, Evaluation Protocol=Full2026.04 | 46.3 | 56.2 | |
| Phi-4-MultimodalSize=5.6B, Evaluation Protocol=Full2026.04 | 46.1 | 59.9 | |
| Qwen3-Omni (I)Size=30B†, Evaluation Protocol=None2026.04 | 44.1 | — | |
| Qwen3-Omni (T)Size=30B†, Evaluation Protocol=None2026.04 | 41.9 | — | |
| Qwen2.5-Omni-7BSize=10.7B, Evaluation Protocol=None2026.04 | 41.3 | — | |
| Qwen3-Omni (I)Size=30B†, Evaluation Protocol=TB2026.04 | 37.6 | — | |
| Audio-Flamingo-3Size=8.4B, Evaluation Protocol=TB2026.04 | 35.3 | — | |
| Qwen2.5-Omni-7BSize=10.7B, Evaluation Protocol=TB2026.04 | 35.3 | — | |
| Voxtral-Mini-3BSize=4.7B, Evaluation Protocol=None2026.04 | 33.8 | — | |
| Audio-Flamingo-3Size=8.4B, Evaluation Protocol=None2026.04 | 33.1 | — | |
| Qwen3-Omni (T)Size=30B†, Evaluation Protocol=TB2026.04 | 31.6 | — | |
| Phi-4-MultimodalSize=5.6B, Evaluation Protocol=TB2026.04 | 28.3 | — | |
| Phi-4-MultimodalSize=5.6B, Evaluation Protocol=None2026.04 | 27.6 | — | |
| Voxtral-Mini-3BSize=4.7B, Evaluation Protocol=TB2026.04 | 26.3 | — | |
| DeSTA-2.5Size=8.8B, Evaluation Protocol=TB2026.04 | 26.2 | — | |
| DeSTA-2.5Size=8.8B, Evaluation Protocol=None2026.04 | 26.1 | — | |
| Qwen2-Audio-7BSize=8.2B, Evaluation Protocol=None2026.04 | 26 | — | |
| Chance LevelEvaluation Protocol=Full2026.04 | 25 | — | |
| Chance LevelEvaluation Protocol=None2026.04 | 25 | — | |
| Chance LevelEvaluation Protocol=TB2026.04 | 25 | — | |
| Qwen2-Audio-7BSize=8.2B, Evaluation Protocol=TB2026.04 | 22.5 | — |