Music Temporal Understanding on MUSTBENCH
62.5TSG Onset Hit@3sQwen 3 Omni
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen 3 OmniSize=30B-A3B2026.05 | 62.5 | 9.5 | 36 | 53.4 | 7.1 | 24.9 | 16 | 63.6 | 8.8 | 12 | 10.4 | 20.3 | 30.2 | |
| Gemini 2.5 FlashSize=–2026.05 | 60 | 71.5 | 65.8 | 56.7 | 11.2 | 34.5 | 22.9 | 42.4 | 24.8 | 33.1 | 29 | 30 | 41.8 | |
| Gemini 3 ProSize=–2026.05 | 60 | 44.5 | 52.3 | 54.3 | 6.5 | 33 | 19.8 | 27.3 | 29.2 | 37.6 | 33.4 | 30.2 | 36.6 | |
| Gemini 2.5 ProSize=–2026.05 | 57.5 | 57 | 57.3 | 62.5 | 10 | 38.1 | 24.1 | 46 | 22.4 | 28.6 | 25.5 | 28.4 | 40.3 | |
| MUSTSize=7B2026.05 | 55.5 | 62.5 | 59 | 60.6 | 21 | 34.6 | 27.8 | 67.2 | 22.6 | 29.1 | 25.9 | 23.3 | 44.1 | |
| Gemini 3 FlashSize=–2026.05 | 55 | 42.5 | 48.8 | 51.4 | 10.7 | 37.9 | 24.3 | 47 | 25.7 | 34.2 | 30 | 27.9 | 38.1 | |
| Music FlamingoSize=8B2026.05 | 53 | 24 | 38.5 | 56.3 | 13.4 | 33.4 | 23.4 | 41.4 | 10.1 | 17.1 | 13.6 | 19.5 | 31.1 | |
| Qwen 2.5 OmniSize=7B2026.05 | 39 | 3.5 | 21.3 | 45.7 | 9.2 | 33.7 | 21.5 | 46.5 | 6.4 | 10.6 | 8.5 | 11.1 | 24.3 | |
| MUSTSize=3B2026.05 | 35.5 | 41 | 38.3 | 58.2 | 21.7 | 35.4 | 28.5 | 57.1 | 24.1 | 31.4 | 27.8 | 21.8 | 38.1 | |
| GPT AudioSize=–2026.05 | 21.5 | 1.5 | 11.5 | 42.3 | 12 | 33 | 22.5 | 36.9 | 13.8 | 19.6 | 16.7 | 21.2 | 22.6 | |
| AF-NextSize=8B2026.05 | 17 | 3 | 10 | 44.7 | 9 | 29.4 | 19.2 | 41.4 | 2.2 | 3.4 | 2.8 | 14.2 | 18.8 | |
| Phi-4-mmSize=6B2026.05 | 14.6 | 0 | 7.3 | 28.4 | 9.6 | 23 | 16.3 | 13.1 | 5 | 8.8 | 6.9 | 10.2 | 12.8 | |
| GPT Audio 1.5Size=–2026.05 | 14 | 3.5 | 8.8 | 51 | 12.4 | 34.8 | 23.6 | 37.9 | 13.7 | 20.3 | 17 | 21.5 | 23.5 | |
| Qwen 2.5 OmniSize=3B2026.05 | 7.5 | 2 | 4.8 | 32.7 | 11 | 33.6 | 22.3 | 37.4 | 8.2 | 12.8 | 10.5 | 10.5 | 18.2 |