Audio Understanding Robustness on MMAU Music SNR=-10dB
64.82AccuracyDFL
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| DFLBackbone=MiniCPM-o-2.62026.05 | 64.82 | 85.4 | 86.57 | |
| EchoDistillBackbone=MiniCPM-o-2.62026.05 | 64.69 | 86.32 | 87.24 | |
| STFTBackbone=MiniCPM-o-2.62026.05 | 64.55 | 86.49 | 87.15 | |
| WTBackbone=MiniCPM-o-2.62026.05 | 64.06 | 85.1 | 86.95 | |
| InitialBackbone=MiniCPM-o-2.62026.05 | 63.63 | — | 85.15 | |
| SEENBackbone=MiniCPM-o-2.62026.05 | 63.63 | 85.47 | 85.05 | |
| STFTBackbone=Step-Audio22026.05 | 60.58 | 78.16 | 79 | |
| EchoDistillBackbone=Step-Audio22026.05 | 59.75 | 78.16 | 86 | |
| WTBackbone=Qwen2.5-Omni2026.05 | 59.33 | 71.33 | 74.16 | |
| InitialBackbone=Step-Audio22026.05 | 59.17 | — | 78.67 | |
| EchoDistillBackbone=Qwen2.5-Omni2026.05 | 59.17 | 74.58 | 75.08 | |
| WTBackbone=Step-Audio22026.05 | 59.08 | 75.08 | 77.33 | |
| DFLBackbone=Step-Audio22026.05 | 59.08 | 74.91 | 75.66 | |
| InitialBackbone=Qwen2.5-Omni2026.05 | 59 | — | 74 | |
| SEENBackbone=Qwen2.5-Omni2026.05 | 58.91 | 74.08 | 73.91 | |
| STFTBackbone=Qwen2.5-Omni2026.05 | 58.75 | 73.91 | 75.41 | |
| SEENBackbone=Step-Audio22026.05 | 57 | 70.75 | 77.08 | |
| DFLBackbone=Qwen2.5-Omni2026.05 | 56.58 | 72.75 | 74.5 |