Audio Understanding Robustness on MMAU Sound (SNR=-10dB)
66.83AccuracyEchoDistill
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| EchoDistillBackbone=Qwen2.5-Omni2026.05 | 66.83 | 72.83 | 75.33 | |
| STFTBackbone=Step-Audio22026.05 | 63.75 | 70.33 | 69.91 | |
| EchoDistillBackbone=Step-Audio22026.05 | 63.67 | 71.75 | 80.08 | |
| InitialBackbone=Step-Audio22026.05 | 63.5 | — | 70.42 | |
| DFLBackbone=Step-Audio22026.05 | 63.5 | 71.5 | 74.58 | |
| WTBackbone=Step-Audio22026.05 | 63.33 | 69 | 70.5 | |
| EchoDistillBackbone=Qwen2.5-Omni2026.05 | 62.96 | 75.36 | 76.14 | |
| EchoDistillBackbone=MiniCPM-o-2.62026.05 | 62.18 | 77.14 | 80.42 | |
| InitialBackbone=MiniCPM-o-2.62026.05 | 61.92 | — | 77.76 | |
| EchoDistillBackbone=MiniCPM-o-2.62026.05 | 61.79 | 79.07 | 81.37 | |
| SEENBackbone=MiniCPM-o-2.62026.05 | 61.56 | 77.2 | 78.04 | |
| DFLBackbone=MiniCPM-o-2.62026.05 | 61.55 | 78.72 | 80.31 | |
| SEENBackbone=MiniCPM-o-2.62026.05 | 61.03 | 78.58 | 78.79 | |
| InitialBackbone=MiniCPM-o-2.62026.05 | 60.94 | — | 79.18 | |
| STFTBackbone=MiniCPM-o-2.62026.05 | 60.93 | 77.11 | 77.78 | |
| WTBackbone=MiniCPM-o-2.62026.05 | 60.86 | 75.27 | 77.66 | |
| STFTBackbone=Qwen2.5-Omni2026.05 | 60.75 | 67.75 | 68.25 | |
| DFLBackbone=MiniCPM-o-2.62026.05 | 60.51 | 78.93 | 80.73 | |
| STFTBackbone=MiniCPM-o-2.62026.05 | 60.37 | 78.47 | 80.01 | |
| SEENBackbone=Step-Audio22026.05 | 59.83 | 67.25 | 77.16 | |
| DFLBackbone=Qwen2.5-Omni2026.05 | 59.75 | 64.66 | 67.41 | |
| EchoDistillBackbone=Step-Audio22026.05 | 59.57 | 73.65 | 80.32 | |
| STFTBackbone=Step-Audio22026.05 | 59.53 | 72.24 | 72.46 | |
| STFTBackbone=Qwen2.5-Omni2026.05 | 59.53 | 71.99 | 72.54 | |
| InitialBackbone=Step-Audio22026.05 | 59.36 | — | 72.75 | |
| SEENBackbone=Qwen2.5-Omni2026.05 | 59.34 | 71.82 | 71.6 | |
| SEENBackbone=Qwen2.5-Omni2026.05 | 59.33 | 66.25 | 66.42 | |
| InitialBackbone=Qwen2.5-Omni2026.05 | 59.32 | — | 71.35 | |
| WTBackbone=MiniCPM-o-2.62026.05 | 58.58 | 76.18 | 78.01 | |
| DFLBackbone=Qwen2.5-Omni2026.05 | 58.26 | 70.02 | 71.35 | |
| InitialBackbone=Qwen2.5-Omni2026.05 | 58.17 | — | 65.5 | |
| DFLBackbone=Step-Audio22026.05 | 58.14 | 70.65 | 71.71 | |
| WTBackbone=Qwen2.5-Omni2026.05 | 56.66 | 61.58 | 63.41 | |
| WTBackbone=Qwen2.5-Omni2026.05 | 56.4 | 66.48 | 68.03 | |
| WTBackbone=Step-Audio22026.05 | 56.34 | 67.38 | 69.11 | |
| SEENBackbone=Step-Audio22026.05 | 56.06 | 68.21 | 74.91 |