Safety Evaluation on Beavertails-V (test)
86.93Helpfulness ScorePragma-VL
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Pragma-VLBackbone=Llava-1.5-7B2026.02 | 86.93 | 88.96 | |
| SFTBackbone=Llava-1.5-7B2026.02 | 80.13 | 80.64 | |
| Beavertails-V helpBackbone=Llava-1.5-7B2026.02 | 79.93 | 65.64 | |
| SAFE RLHF-vBackbone=Llava-1.5-7B2026.02 | 76.74 | 84.55 | |
| SPA-VLBackbone=Llava-1.5-7B2026.02 | 68.93 | 78.27 | |
| MM-RLHFBackbone=Llava-1.5-7B2026.02 | 67.57 | 68.25 | |
| Pragma-VLBackbone=Qwen2.5-VL-7B2026.02 | 62.65 | 67.91 | |
| DPOBackbone=Llava-1.5-7B2026.02 | 60.1 | 72.66 | |
| Beavertails-V harmBackbone=Llava-1.5-7B2026.02 | 57.55 | 71.13 | |
| Beavertails-V allBackbone=Llava-1.5-7B2026.02 | 55.85 | 69.21 | |
| SFTBackbone=Qwen2.5-VL-7B2026.02 | 53.14 | 61.46 | |
| Qwen2.5-VL-7BBackbone=Qwen2.5-VL-7B2026.02 | 50 | 50 | |
| Llava-1.5-7BBackbone=Llava-1.5-7B2026.02 | 50 | 50 | |
| Beavertails-V helpBackbone=Qwen2.5-VL-7B2026.02 | 49.91 | 43.29 | |
| DPOBackbone=Qwen2.5-VL-7B2026.02 | 48.13 | 59.96 | |
| SAFE RLHF-vBackbone=Qwen2.5-VL-7B2026.02 | 46.85 | 57.72 | |
| Beavertails-V allBackbone=Qwen2.5-VL-7B2026.02 | 45.84 | 56.12 | |
| SPA-VLBackbone=Qwen2.5-VL-7B2026.02 | 44.99 | 54.16 | |
| MM-RLHFBackbone=Qwen2.5-VL-7B2026.02 | 37.97 | 51.03 | |
| Beavertails-V harmBackbone=Qwen2.5-VL-7B2026.02 | 37.07 | 48.63 |