Multi-choice Audio Understanding on MMAU speech
201N_VV CountQwen-Omni
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Qwen-OmniBackbone=Streaming-modified Whisper, LLM=Qwen2.5 Thinker, Adapter=Learned projection2025.05 | 201 | 76 | 76 | 22 | 0.91 | 0.9 | 31.1 | 32.1 | |
| AF3Backbone=AF-Whisper, LLM=Qwen2.5-7B2025.05 | 176 | 67 | 66 | 27 | 0.95 | 0.92 | 29.3 | 31.6 | |
| Qwen2-AudioBackbone=Whisper-large-v3, LLM=Qwen-7B, Adapter=None2025.05 | 128 | 54 | 55 | 32 | 0.91 | 0.9 | 30.7 | 31.9 | |
| SALMONNBackbone=Whisper-v2 + BEATs, LLM=Vicuna-13B, Adapter=Q-Former2025.05 | 102 | 41 | 53 | 35 | 1.1 | 1.1 | 14.8 | 14.7 |