Harmfulness Evaluation on SORRY-Bench audio
78.41ASR AccuracyMDSteer-c2r
Evaluation Results
| Method | Links | |
|---|---|---|
| MDSteer-c2rModel=Qwen2-Audio2025.10 | 78.41 | |
| MDSteer-h2sModel=Qwen2-Audio2025.10 | 75.45 | |
| MDSteer-h2sModel=Kimi-Audio2025.10 | 55 | |
| RRSDefense runtime (s)=11452025.10 | 38.41 | |
| No DefenseModel=Qwen2-Audio2025.10 | 27.5 | |
| No Defense2025.10 | 27.5 | |
| MDSteer-c2rModel=Kimi-Audio2025.10 | 21.59 | |
| AdaShieldModel=Qwen2-Audio2025.10 | 20.45 | |
| SARSteerModel=Qwen2-Audio2025.10 | 13.41 | |
| SARSteerDefense runtime (s)=2662025.10 | 13.41 | |
| No DefenseModel=Kimi-Audio2025.10 | 12.5 | |
| FSDModel=Kimi-Audio2025.10 | 11.14 | |
| FSDModel=Qwen2-Audio2025.10 | 10.55 | |
| SARSteerModel=Kimi-Audio2025.10 | 6.14 | |
| AdaShieldModel=Kimi-Audio2025.10 | 0.23 |