Safety Evaluation on Omni-SafetyBench
3.5ASR (Image Avg)Refusal Steering
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Refusal SteeringBase Model=Qwen2.5-Omni-3B2026.05 | 3.5 | 1.6 | 0.2 | 3.6 | 7.6 | 3.3 | |
| Refusal SteeringBase Model=MiniCPM-o-4.52026.05 | 6.5 | 11.6 | 50.6 | 14.6 | 36.6 | 19 | |
| ReGapBase Model=Qwen2.5-Omni-3B2026.05 | 17.1 | 8.9 | 25.9 | 25.8 | 34 | 20.8 | |
| ReGapBase Model=MiniCPM-o-4.52026.05 | 30.6 | 14.2 | 16.6 | 9.1 | 63 | 27.8 | |
| ReGapBase Model=Qwen3-Omni-30B-A3B-Instruct2026.05 | 48.5 | 8.8 | 14.1 | 11.8 | 12 | 27.5 | |
| Refusal SteeringBase Model=Qwen3-Omni-30B-A3B-Instruct2026.05 | 49.8 | 2.6 | 6 | 6.2 | 14.4 | 25.5 | |
| BaseBase Model=Qwen2.5-Omni-3B2026.05 | 82.1 | 56.6 | 48.8 | 43 | 65.6 | 65.7 | |
| BaseBase Model=Qwen3-Omni-30B-A3B-Instruct2026.05 | 83.8 | 20.8 | 22.4 | 12.6 | 74.4 | 54.5 | |
| BaseBase Model=MiniCPM-o-4.52026.05 | 86.4 | 57.6 | 35.2 | 24 | 72.6 | 64.1 |