Safety Evaluation on SPA-VL (test)
97.93Helpfulness ScorePragma-VL
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Pragma-VLBackbone=Llava-1.5-7B2026.02 | 97.93 | 92.05 | |
| SFTBackbone=Llava-1.5-7B2026.02 | 89.91 | 86.79 | |
| Pragma-VLBackbone=Qwen2.5-VL-7B2026.02 | 87.17 | 87.92 | |
| Beavertails-V helpBackbone=Llava-1.5-7B2026.02 | 80.27 | 57.74 | |
| SPA-VLBackbone=Llava-1.5-7B2026.02 | 73.3 | 86.79 | |
| DPOBackbone=Llava-1.5-7B2026.02 | 69.33 | 93.96 | |
| SAFE RLHF-vBackbone=Llava-1.5-7B2026.02 | 68.48 | 78.87 | |
| SFTBackbone=Qwen2.5-VL-7B2026.02 | 63.64 | 64.91 | |
| MM-RLHFBackbone=Llava-1.5-7B2026.02 | 62.5 | 66.79 | |
| Beavertails-V allBackbone=Llava-1.5-7B2026.02 | 61.51 | 65.28 | |
| Beavertails-V harmBackbone=Llava-1.5-7B2026.02 | 56.7 | 81.13 | |
| DPOBackbone=Qwen2.5-VL-7B2026.02 | 52.47 | 78.87 | |
| Qwen2.5-VL-7BBackbone=Qwen2.5-VL-7B2026.02 | 50 | 50 | |
| Llava-1.5-7BBackbone=Llava-1.5-7B2026.02 | 50 | 50 | |
| SAFE RLHF-vBackbone=Qwen2.5-VL-7B2026.02 | 45.08 | 61.51 | |
| Beavertails-V helpBackbone=Qwen2.5-VL-7B2026.02 | 40.47 | 29.43 | |
| Beavertails-V allBackbone=Qwen2.5-VL-7B2026.02 | 37.71 | 51.69 | |
| Beavertails-V harmBackbone=Qwen2.5-VL-7B2026.02 | 26.88 | 45.66 | |
| SPA-VLBackbone=Qwen2.5-VL-7B2026.02 | 26.79 | 46.04 | |
| MM-RLHFBackbone=Qwen2.5-VL-7B2026.02 | 20.31 | 48.09 |