Audio Understanding Robustness on MMAU Speech SNR=-10dB
62.89AccuracyEchoDistill
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| EchoDistillBackbone=Qwen2.5-Omni2026.05 | 62.89 | 78.67 | 78.01 | |
| InitialBackbone=Qwen2.5-Omni2026.05 | 60.8 | — | 74.56 | |
| SEENBackbone=Qwen2.5-Omni2026.05 | 59.79 | 75.14 | 74.47 | |
| STFTBackbone=Qwen2.5-Omni2026.05 | 59.09 | 74.3 | 73.97 | |
| EchoDistillBackbone=MiniCPM-o-2.62026.05 | 58.49 | 73.75 | 76.46 | |
| DFLBackbone=Qwen2.5-Omni2026.05 | 58.46 | 72.65 | 72.15 | |
| SEENBackbone=MiniCPM-o-2.62026.05 | 57.91 | 73.07 | 73.27 | |
| InitialBackbone=MiniCPM-o-2.62026.05 | 57.27 | — | 74.64 | |
| STFTBackbone=MiniCPM-o-2.62026.05 | 55.62 | 71.8 | 75.11 | |
| InitialBackbone=Step-Audio22026.05 | 55.42 | — | 69.15 | |
| EchoDistillBackbone=Step-Audio22026.05 | 55.29 | 71.04 | 74.89 | |
| DFLBackbone=MiniCPM-o-2.62026.05 | 55.17 | 72.66 | 75.31 | |
| STFTBackbone=Step-Audio22026.05 | 54.25 | 68.22 | 68.47 | |
| WTBackbone=Qwen2.5-Omni2026.05 | 53.21 | 66.52 | 66.52 | |
| DFLBackbone=Step-Audio22026.05 | 51.83 | 65.55 | 64.88 | |
| SEENBackbone=Step-Audio22026.05 | 51.34 | 66.63 | 70.48 | |
| WTBackbone=MiniCPM-o-2.62026.05 | 50.82 | 68.16 | 69.43 | |
| WTBackbone=Step-Audio22026.05 | 46.62 | 58.07 | 59.49 |