Safety Evaluation on MM-SafetyBench (test)
68.95Helpfulness ScoreBeavertails-V help
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Beavertails-V helpBackbone=Llava-1.5-7B2026.02 | 68.95 | 58.22 | 59.14 | — | |
| Pragma-VLBackbone=Llava-1.5-7B2026.02 | 68.37 | 67.78 | 31.67 | — | |
| DPOBackbone=Llava-1.5-7B2026.02 | 57.1 | 60.69 | 43.4 | — | |
| Beavertails-V helpBackbone=Qwen2.5-VL-7B2026.02 | 54.94 | 52.38 | 51.07 | — | |
| Pragma-VLBackbone=Qwen2.5-VL-7B2026.02 | 52.74 | 58.99 | 31.66 | — | |
| SFTBackbone=Llava-1.5-7B2026.02 | 51.36 | 56.07 | 41.38 | — | |
| Qwen2.5-VL-7BBackbone=Qwen2.5-VL-7B2026.02 | 50 | 50 | 48.75 | — | |
| Llava-1.5-7BBackbone=Llava-1.5-7B2026.02 | 50 | 50 | 56.49 | — | |
| SPA-VLBackbone=Llava-1.5-7B2026.02 | 47.26 | 54.17 | 44.39 | — | |
| Beavertails-V allBackbone=Llava-1.5-7B2026.02 | 47.02 | 52.9 | 51.53 | — | |
| SAFE RLHF-vBackbone=Qwen2.5-VL-7B2026.02 | 45.18 | 53.95 | 43.2 | — | |
| SAFE RLHF-vBackbone=Llava-1.5-7B2026.02 | 44.69 | 53.27 | 48.56 | — | |
| SFTBackbone=Qwen2.5-VL-7B2026.02 | 43.29 | 53.36 | 39.07 | — | |
| DPOBackbone=Qwen2.5-VL-7B2026.02 | 39.66 | 51.97 | 36.79 | — | |
| Beavertails-V allBackbone=Qwen2.5-VL-7B2026.02 | 38.97 | 51.68 | 49.58 | — | |
| MM-RLHFBackbone=Llava-1.5-7B2026.02 | 37.44 | 43.69 | 46.93 | — | |
| SPA-VLBackbone=Qwen2.5-VL-7B2026.02 | 35.43 | 49.26 | 48.24 | — | |
| Beavertails-V harmBackbone=Qwen2.5-VL-7B2026.02 | 27.44 | 45.18 | 43.29 | — | |
| Beavertails-V harmBackbone=Llava-1.5-7B2026.02 | 25.95 | 38.8 | 40.77 | — | |
| MM-RLHFBackbone=Qwen2.5-VL-7B2026.02 | 20.16 | 32.92 | 35.62 | — | |
| BaseBackbone=Qwen2.5-Omni-3B2026.05 | — | — | — | 76.8 | |
| BaseBackbone=MiniCPM-o-4.52026.05 | — | — | — | 69.8 | |
| BaseBackbone=Qwen3-Omni-30B-A3B-Instruct2026.05 | — | — | — | 58.1 | |
| ReGapBackbone=Qwen2.5-Omni-3B2026.05 | — | — | — | 14.2 | |
| ReGapBackbone=MiniCPM-o-4.52026.05 | — | — | — | 65.4 | |
| ReGapBackbone=Qwen3-Omni-30B-A3B-Instruct2026.05 | — | — | — | 36.3 | |
| ShiftDCBackbone=Qwen2.5-Omni-3B2026.05 | — | — | — | 46.3 | |
| ShiftDCBackbone=MiniCPM-o-4.52026.05 | — | — | — | 57.6 | |
| ShiftDCBackbone=Qwen3-Omni-30B-A3B-Instruct2026.05 | — | — | — | 26.9 | |
| SPA-VL-DPOBackbone=Qwen2.5-Omni-3B2026.05 | — | — | — | 67.4 | |
| VLGuard-SFTBackbone=Qwen2.5-Omni-3B2026.05 | — | — | — | 36.2 |