Music Understanding on MuChoMusic
85.7CC ScoreGemini-3.0 Pro
Evaluation results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Gemini-3.0 ProParam.=-2026.05 | 85.7 | 91.4 | 90 | 88.3 | 76.8 | 81 | 84.2 | 85.7 | 84.8 | 76.3 | 82.2 | 83.8 | 76.2 | 79.8 | |
| Gemini-2.5 ProParam.=-2026.05 | 83.3 | 94.1 | 94.6 | 87.7 | 78.9 | 80.4 | 100 | 88 | 70.7 | 87.8 | 83.3 | 76.9 | 78 | 79.4 | |
| Kimi-AudioParam.=8B2026.05 | 82.1 | 77.1 | 70 | 77.9 | 74.8 | 68.3 | 57.9 | 78.6 | 63 | 65.8 | 68.9 | 67.1 | 66.2 | 68.2 | |
| Audio-Flamingo3Param.=8B2026.05 | 78.6 | 91.4 | 85 | 68.8 | 66.7 | 75.6 | 57.9 | 92.9 | 76.1 | 77.6 | 76.7 | 73.4 | 73.9 | 73.4 | |
| GaMMA-14BParam.=14B2026.05 | 78.6 | 88.6 | 90 | 85.7 | 78.3 | 79.5 | 84.2 | 82.1 | 82.6 | 77.6 | 77.8 | 81.5 | 77.2 | 79 | |
| GaMMA-8BParam.=8B2026.05 | 75 | 88.6 | 90 | 83.1 | 77.3 | 76.6 | 78.9 | 78.6 | 82.6 | 76.3 | 77.8 | 78 | 78.1 | 78 | |
| Gemini-2.0-ProParam.=-2026.05 | 63.7 | 60 | 65 | 66.2 | 59.1 | 65.9 | 68.4 | 82.1 | 73.9 | 71.1 | 57.8 | 81.1 | 64.6 | 63.8 | |
| Qwen2.5-OmniParam.=8B2026.05 | 61 | 86 | 88 | 82 | 78 | 78 | 68 | 86 | 72 | 75 | 76 | 69 | 78 | 76 | |
| M2UGenParam.=7B2026.05 | 50 | 25.7 | 40 | 33.8 | 41.9 | 42.4 | 36.8 | 53.6 | 54.4 | 36.8 | 44.4 | 44.5 | 49.2 | 43.9 | |
| QwenAudioParam.=7B2026.05 | 46.8 | 42.9 | 40 | 46.8 | 49.5 | 52.7 | 26.3 | 60.7 | 52.2 | 34.2 | 48.9 | 52.6 | 50.4 | 49.1 | |
| Qwen2AudioParam.=7B2026.05 | 42.9 | 54.3 | 65 | 61 | 53.3 | 64.4 | 52.6 | 64.3 | 50 | 64.5 | 71.1 | 66.5 | 64.6 | 61.5 | |
| SALMONNParam.=13B2026.05 | 39.3 | 17.1 | 37.5 | 37.7 | 37.9 | 42.9 | 42.1 | 50 | 34.8 | 32.9 | 50 | 35.3 | 38.2 | 37 | |
| Mu-LLaMAParam.=7B2026.05 | 25 | 40 | 22.5 | 23.4 | 33.3 | 25.4 | 15.8 | 21.4 | 37 | 31.6 | 28.9 | 29.5 | 33.3 | 30.6 |