Multimodal Audio Reasoning on MMAR
63.5Mean ScoreGPT-4o Audio
Evaluation Results
| Method | Links | |
|---|---|---|
| GPT-4o Audio2026.03 | 63.5 | |
| Audio Flamingo 3Size=8B, No LLM Modification=×, # Samples=26.7M, # Hours=54.4K2026.03 | 58.5 | |
| Qwen2.5-OmniSize=7B, No LLM Modification=×, # Tokens=0.3T2026.03 | 56.7 | |
| DeSTA-2.5-AudioSize=8B, No LLM Modification=✓, # Samples=5M, # Hours=7K2026.03 | 50.8 | |
| GPT-4o mini Audio2026.03 | 50.6 | |
| AL-Whisper-Instruct-RSize=4B, No LLM Modification=✓, # Samples=0.5M, # Hours=1K, # Tokens=91M2026.03 | 49.1 | |
| ALARM-ESize=4B, No LLM Modification=✓, # Samples=5.5M, # Hours=17K, # Tokens=1.5B2026.03 | 48.7 | |
| ALARM-PSize=4B, No LLM Modification=✓, # Samples=5.5M, # Hours=17K, # Tokens=1.5B2026.03 | 43.5 | |
| ALARM-CASize=4B, No LLM Modification=✓, # Samples=5.5M, # Hours=17K, # Tokens=1.5B2026.03 | 40.6 | |
| AL-W2VBERT2-RSize=4B, No LLM Modification=✓, # Samples=0.5M, # Hours=0.6K, # Tokens=58M2026.03 | 37.4 | |
| AL-Whisper-RSize=4B, No LLM Modification=✓, # Samples=5.5M, # Hours=17K, # Tokens=1.5B2026.03 | 37.3 | |
| Audio ReasonerSize=8B, No LLM Modification=×, # Samples=> 1.2M, # Hours=> 320K2026.03 | 36.8 | |
| AL-MuQ-RSize=4B, No LLM Modification=✓, # Samples=0.4M, # Hours=1.5K, # Tokens=137M2026.03 | 36 | |
| AL-SSLAM-RSize=4B, No LLM Modification=✓, # Samples=0.4M, # Hours=1.2K, # Tokens=108M2026.03 | 33.8 | |
| SALMONNSize=13B, No LLM Modification=×, # Samples=2.3M, # Hours=4.4K2026.03 | 33.2 | |
| Qwen2-Audio-InstructSize=7B, No LLM Modification=×, # Hours=320K2026.03 | 30 | |
| GAMASize=7B, No LLM Modification=×, # Samples=6.3M2026.03 | 26.5 | |
| Audio Flamingo 2Size=3B, No LLM Modification=×, # Samples=5.9M2026.03 | 21.9 | |
| LTUSize=7B, No LLM Modification=×, # Samples=5.6M2026.03 | 19.2 | |
| GAMA-ITSize=7B, No LLM Modification=×, # Samples=6.5M2026.03 | 17.4 | |
| MU-LLaMASize=7B, No LLM Modification=✓, # Samples=0.1M, # Hours=762026.03 | 13.9 | |
| MusiLingoSize=7B, No LLM Modification=✓, # Samples=0.7M2026.03 | 6.6 |