Safety Evaluation on AudioJailBench (AJB)
1.4ASR Error Rate (Audio)Refusal Steering
Evaluation results
| Method | Links | |
|---|---|---|
| Refusal SteeringBase Model=Qwen2.5-Omni-3B2026.05 | 1.4 | |
| ReGapBase Model=MiniCPM-o-4.52026.05 | 9.2 | |
| Refusal SteeringBase Model=Qwen3-Omni-30B-A3B-Instruct2026.05 | 12.4 | |
| ReGapBase Model=Qwen3-Omni-30B-A3B-Instruct2026.05 | 16.9 | |
| ReGapBase Model=Qwen2.5-Omni-3B2026.05 | 17 | |
| Refusal SteeringBase Model=MiniCPM-o-4.52026.05 | 19.9 | |
| BaseBase Model=Qwen3-Omni-30B-A3B-Instruct2026.05 | 26.4 | |
| BaseBase Model=MiniCPM-o-4.52026.05 | 45.6 | |
| BaseBase Model=Qwen2.5-Omni-3B2026.05 | 49.6 |