Multimodal Safety Evaluation on FigStep
0.21ASR (%)CrossGuard
Evaluation results
| Method | Links | |
|---|---|---|
| CrossGuardCategory=MLLM Guardrails2025.10 | 0.21 | |
| GPT-4oCategory=Online MLLMs2025.10 | 1.6 | |
| JailDAMCategory=MLLM Guardrails2025.10 | 6 | |
| Claude-3.5-SonnetCategory=Online MLLMs2025.10 | 13 | |
| Qwen2.5-VL-7BCategory=Offline MLLMs2025.10 | 24.2 | |
| LLaVA-1.5-7B (base)Category=Offline MLLMs2025.10 | 62.6 | |
| Llama-Guard3-VisionCategory=MLLM Guardrails2025.10 | 66.92 | |
| HiddenDetectCategory=MLLM Guardrails2025.10 | 72.2 | |
| LlavaGuardCategory=MLLM Guardrails2025.10 | 83.08 |