Safety Evaluation on JailBreakV
0.72ASRCrossGuard
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| CrossGuardCategory=MLLM Guardrails2025.10 | 0.72 | — | — | — | — | |
| Qwen2.5-VL-7BCategory=Offline MLLMs2025.10 | 2.14 | — | — | — | — | |
| HiddenDetectCategory=MLLM Guardrails2025.10 | 4.64 | — | — | — | — | |
| Claude-3.5-SonnetCategory=Online MLLMs2025.10 | 5 | — | — | — | — | |
| GPT-4oCategory=Online MLLMs2025.10 | 6.08 | — | — | — | — | |
| CAREBackbone=Qwen2.5-VL2026.03 | 6.55 | — | — | — | — | |
| ShiftDCBackbone=Qwen2.5-VL2026.03 | 11.17 | — | — | — | — | |
| AM3SafetyBackbone=Qwen2.5-VL-7B2026.01 | 11.87 | — | — | 63.71 | 68.14 | |
| ProFSBackbone=Qwen2.5-VL2026.03 | 16.3 | — | — | — | — | |
| SPA-VLBackbone=Qwen2.5-VL-7B2026.01 | 21.46 | — | — | 28.1 | 53.81 | |
| WILDBREAKBase Model=Mistral-7B, Alignment=WILDBREAK2025.05 | 24 | 14 | 2.43 | — | — | |
| TRIDENT-EDGEBase Model=Mistral-7B, Alignment=TRIDENT-EDGE2025.05 | 26 | 27 | 2.32 | — | — | |
| JailDAMCategory=MLLM Guardrails2025.10 | 32.5 | — | — | — | — | |
| MM-DPOBackbone=Qwen2.5-VL-7B2026.01 | 33.79 | — | — | 53.92 | 55.45 | |
| Llama-Guard3-VisionCategory=MLLM Guardrails2025.10 | 34.29 | — | — | — | — | |
| AM3SafetyBackbone=LLaVA-NeXT-7B2026.01 | 35.16 | — | — | 68.47 | 70.56 | |
| Mistral-7B (Unaligned)Base Model=Mistral-7B, Alignment=None (Unaligned)2025.05 | 43 | 33 | 3.13 | — | — | |
| Safe RLHF-VBackbone=Qwen2.5-VL-7B2026.01 | 43.84 | — | — | 62.21 | 56.96 | |
| Qwen2.5-VL-7BAlignment=Original Base Model2026.01 | 44.29 | — | — | 50 | 50 | |
| LLaVA-1.5-7B (base)Category=Offline MLLMs2025.10 | 51.43 | — | — | — | — | |
| SPA-VLBackbone=LLaVA-NeXT-7B2026.01 | 52.05 | — | — | 17.01 | 63.4 | |
| RLHF-VBackbone=Qwen2.5-VL-7B2026.01 | 54.33 | — | — | 31.47 | 50.16 | |
| LLaVA-NeXT-7BAlignment=Original Base Model2026.01 | 74.89 | — | — | 50 | 50 | |
| MM-DPOBackbone=LLaVA-NeXT-7B2026.01 | 75.8 | — | — | 50.72 | 49.03 | |
| Safe RLHF-VBackbone=LLaVA-NeXT-7B2026.01 | 76.26 | — | — | 46 | 50.43 | |
| RLHF-VBackbone=LLaVA-NeXT-7B2026.01 | 78.08 | — | — | 25.76 | 48.35 | |
| LlavaGuardCategory=MLLM Guardrails2025.10 | 90.71 | — | — | — | — |