Reasoning on VoiceBench
72.9MMSU Accuracy (Audio)GPT-4o-mini-Audio
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| GPT-4o-mini-AudioBackbone=GPT-4o-mini, System Type=Proprietary2026.01 | 72.9 | 81.23 | 84.84 | 90.11 | 78.87 | 85.67 | 92.06 | |
| TARS (Phi-4-MM)Backbone=Phi-4-7B, System Type=Base & Aligned Models2026.01 | 70.14 | 75.76 | 89.45 | 91.87 | 79.8 | 83.82 | 100.45 | |
| AlignChatBackbone=Qwen2.5-7B, System Type=Existing Baseline, Frozen LLM Backbone=true2026.01 | 69.65 | 71.65 | 85.49 | 83.74 | 77.57 | 77.7 | 99.83 | |
| ASR + Phi-4-7BBackbone=Phi-4-7B, System Type=Cascaded2026.01 | 69 | 74.92 | 77.8 | 83.96 | 73.4 | 79.44 | 92.4 | |
| TARS (Qwen2.5-Omni)Backbone=Qwen2.5-7B, System Type=Base & Aligned Models2026.01 | 67.96 | 68.54 | 85.71 | 88.57 | 76.84 | 78.56 | 98.89 | |
| ASR + Qwen2.5-7BBackbone=Qwen2.5-7B, System Type=Cascaded2026.01 | 67.1 | 71.65 | 84 | 83.74 | 75.55 | 77.7 | 97.23 | |
| Knowledge DistillationBackbone=Qwen2.5-7B, System Type=Existing Baseline2026.01 | 63.09 | 69.15 | 82.64 | 84.62 | 72.87 | 76.89 | 93.78 | |
| Qwen2.5-OmniBackbone=Qwen2.5-7B, System Type=Base & Aligned Models2026.01 | 61.51 | 67.94 | 81.09 | 84.4 | 71.3 | 76.17 | 91.76 | |
| DeSTA2.5-AudioBackbone=Llama3.1-8B, System Type=Existing Baseline, Frozen LLM Backbone=true2026.01 | 60.87 | 65.65 | 74.06 | 80.88 | 67.47 | 73.27 | 92.08 | |
| ASR + Llama3.1-8BBackbone=Llama3.1-8B, System Type=Cascaded2026.01 | 58.78 | 65.65 | 72.53 | 80.88 | 65.66 | 73.27 | 89.61 | |
| SALAD-7BBackbone=Qwen2.5-7B, System Type=Existing Baseline2026.01 | 57.5 | 71.6 | 75.1 | 90.1 | 66.3 | 80.85 | 85.33 | |
| Phi-4-MMBackbone=Phi-4-7B, System Type=Base & Aligned Models2026.01 | 54.81 | 72.15 | 71.65 | 84.62 | 63.23 | 78.39 | 79.59 | |
| MiniCPM-o 2.6Backbone=Qwen2.5-7B, System Type=Existing Baseline2026.01 | 54.78 | 59.42 | 78.02 | 82.86 | 66.4 | 71.14 | 85.46 |