Multi-audio Multiple-choice Question Answering on MUGEN
90.67S&PGemini-3-pro
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Gemini-3-proThinking level=High2026.03 | 90.67 | 71.5 | 77.5 | 53.6 | 56.8 | 68.4 | 72.8 | 69.6 | |
| Gemini-3-proThinking level=Low2026.03 | 89.33 | 68 | 77 | 48 | 55.6 | 69.6 | 69.2 | 67.66 | |
| ASR + LLMThinking level=High2026.03 | 82.67 | 16.5 | 27.75 | 28.8 | 26 | 26 | 22.4 | 30.06 | |
| ASR + LLMThinking level=Low2026.03 | 75.33 | 20 | 27 | 28.8 | 26.4 | 26 | 18 | 29.09 | |
| Qwen2.5-OmniType=Open-source LALM2026.03 | 70 | 12 | 32.5 | 15.2 | 10.8 | 53.2 | 18 | 28.69 | |
| Voxtral-Small-24BType=Open-source LALM2026.03 | 64.67 | 27.5 | 32.5 | 22.8 | 22.4 | 25.6 | 16.8 | 28.63 | |
| Voxtral-Mini-3BType=Open-source LALM2026.03 | 59.33 | 25.5 | 30 | 20 | 22.4 | 24.8 | 23.6 | 27.83 | |
| Phi-4-multimodal-instructType=Open-source LALM2026.03 | 52 | 21 | 21 | 28 | 24.8 | 24.8 | 24.8 | 26.29 | |
| DeSTA2.5-AudioType=Open-source LALM2026.03 | 46.67 | 22 | 21.75 | 17.2 | 30.4 | 18 | 28.4 | 24.91 | |
| Audio Flamingo 3Type=Open-source LALM2026.03 | 25.33 | 22.5 | 15 | 21.2 | 23.2 | 1.2 | 19.2 | 17.43 |