Safety Evaluation on MSSbench (test)
99.66Effectiveness ScorePragma-VL
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Pragma-VLBackbone=Qwen2.5-VL-7B2026.02 | 99.66 | 55.89 | |
| Beavertails-V harmBackbone=Qwen2.5-VL-7B2026.02 | 99.15 | 50.5 | |
| Beavertails-V helpBackbone=Qwen2.5-VL-7B2026.02 | 98.98 | 32.54 | |
| SAFE RLHF-vBackbone=Qwen2.5-VL-7B2026.02 | 98.98 | 52.2 | |
| Beavertails-V helpBackbone=Llava-1.5-7B2026.02 | 98.82 | 31.19 | |
| Beavertails-V allBackbone=Qwen2.5-VL-7B2026.02 | 98.65 | 45.45 | |
| DPOBackbone=Qwen2.5-VL-7B2026.02 | 98.65 | 53.96 | |
| Pragma-VLBackbone=Llava-1.5-7B2026.02 | 98.65 | 55.05 | |
| Qwen2.5-VL-7BBackbone=Qwen2.5-VL-7B2026.02 | 98.48 | 36.53 | |
| SPA-VLBackbone=Qwen2.5-VL-7B2026.02 | 98.48 | 36.36 | |
| DPOBackbone=Llava-1.5-7B2026.02 | 97.47 | 47.89 | |
| SPA-VLBackbone=Llava-1.5-7B2026.02 | 97.3 | 28.78 | |
| MM-RLHFBackbone=Qwen2.5-VL-7B2026.02 | 97.13 | 43.09 | |
| Llava-1.5-7BBackbone=Llava-1.5-7B2026.02 | 97.13 | 28.11 | |
| MM-RLHFBackbone=Llava-1.5-7B2026.02 | 97.13 | 37.03 | |
| Beavertails-V allBackbone=Llava-1.5-7B2026.02 | 96.97 | 43.09 | |
| SFTBackbone=Qwen2.5-VL-7B2026.02 | 96.13 | 45.28 | |
| SFTBackbone=Llava-1.5-7B2026.02 | 96.12 | 35.97 | |
| SAFE RLHF-vBackbone=Llava-1.5-7B2026.02 | 95.95 | 44.26 | |
| Beavertails-V harmBackbone=Llava-1.5-7B2026.02 | 87.54 | 40.9 |