Multimodal Safety on Multimodal Safety Suite Avg
90.92Macro-average F1SingGuard-8B
Evaluation results
| Method | Links | |
|---|---|---|
| SingGuard-8BCategory=Our Models2026.06 | 90.92 | |
| SingGuard-4BCategory=Our Models2026.06 | 89.45 | |
| SingGuard-2BCategory=Our Models2026.06 | 89.24 | |
| LLaVAShieldCategory=Open-Source Models2026.06 | 88.42 | |
| GPT-5.1Category=Closed-Source Models2026.06 | 83.49 | |
| Qwen3-VL-235BCategory=Open-Source Models2026.06 | 81.67 | |
| Gemini3-ProCategory=Closed-Source Models2026.06 | 76.58 | |
| ShieldGemma-2Category=Open-Source Models2026.06 | 74.28 | |
| Qwen3-VL-8BCategory=Open-Source Models2026.06 | 64.93 | |
| GuardReasoner-VL-7BCategory=Open-Source Models2026.06 | 64.84 | |
| Qwen3-VL-4BCategory=Open-Source Models2026.06 | 63.69 | |
| SafeGuard-VLCategory=Open-Source Models2026.06 | 63.53 | |
| LlamaGuard4Category=Open-Source Models2026.06 | 50.25 | |
| LlavaGuardCategory=Open-Source Models2026.06 | 49.4 | |
| LlamaGuard3-Vision-11BCategory=Open-Source Models2026.06 | 40.95 |