Large Audio-Language Model Safety Evaluation on Figstep audio
62.8ASRMDSteer-h2s
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| MDSteer-h2sModel=Qwen3-Omni-30B (MoE)2025.10 | 62.8 | 65.6 | 0 | |
| MDSteer-h2sModel=Qwen2.5-Omni-7B2025.10 | 58.8 | 50.8 | 0 | |
| MDSteer-c2rModel=Qwen2.5-Omni-7B2025.10 | 52.3 | 53.8 | 0.8 | |
| MDSteer-h2sModel=Voxtral-Mini-3B2025.10 | 50.8 | 65.4 | 0 | |
| No DefenseModel=Qwen2.5-Omni-7B2025.10 | 42 | 71.8 | 0 | |
| FSDModel=Voxtral-Mini-3B2025.10 | 35.2 | 65.8 | 12 | |
| No DefenseModel=Voxtral-Mini-3B2025.10 | 34 | 71.4 | 0.4 | |
| SARSteerModel=Qwen2.5-Omni-7B2025.10 | 31.2 | 80.2 | 0.8 | |
| SARSteerModel=Voxtral-Mini-3B2025.10 | 30 | 69.2 | 0.8 | |
| FSDModel=Qwen2.5-Omni-7B2025.10 | 28.2 | 73.5 | 10.8 | |
| MDSteer-c2rModel=Voxtral-Mini-3B2025.10 | 26.8 | 67.6 | 0.8 | |
| AdaShieldModel=Qwen2.5-Omni-7B2025.10 | 25.2 | 68.6 | 54.4 | |
| MDSteer-c2rModel=Qwen3-Omni-30B (MoE)2025.10 | 15.2 | 73.4 | 2 | |
| No DefenseModel=Qwen3-Omni-30B (MoE)2025.10 | 7.2 | 95.3 | 1 | |
| FSDModel=Qwen3-Omni-30B (MoE)2025.10 | 6.4 | 89.8 | 12.4 | |
| SARSteerModel=Qwen3-Omni-30B (MoE)2025.10 | 5.2 | 94 | 1.6 | |
| AdaShieldModel=Qwen3-Omni-30B (MoE)2025.10 | 3.6 | 70.6 | 55.2 | |
| AdaShieldModel=Voxtral-Mini-3B2025.10 | 2.4 | 66.4 | 54 |