Safety Evaluation on AudioJailBench (AJB) (test)
49.6ASR Accuracy (Audio)Base
Evaluation Results
| Method | Links | |
|---|---|---|
| BaseBackbone=Qwen2.5-Omni-3B2026.05 | 49.6 | |
| BaseBackbone=MiniCPM-o-4.52026.05 | 45.6 | |
| SPA-VL-DPOBackbone=Qwen2.5-Omni-3B2026.05 | 31.8 | |
| BaseBackbone=Qwen3-Omni-30B-A3B-Instruct2026.05 | 26.4 | |
| ShiftDCBackbone=Qwen2.5-Omni-3B2026.05 | 22.2 | |
| VLGuard-SFTBackbone=Qwen2.5-Omni-3B2026.05 | 20.8 | |
| ShiftDCBackbone=Qwen3-Omni-30B-A3B-Instruct2026.05 | 18.2 | |
| ReGapBackbone=Qwen2.5-Omni-3B2026.05 | 17 | |
| ReGapBackbone=Qwen3-Omni-30B-A3B-Instruct2026.05 | 16.9 | |
| ReGapBackbone=MiniCPM-o-4.52026.05 | 9.2 | |
| ShiftDCBackbone=MiniCPM-o-4.52026.05 | 6.6 |