Visual Compliance Verification on UnsafeBench
76Unsafe F1CompAgent
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| CompAgentCategory=Proposed2025.10 | 76 | 82 | 70 | 87 | 81 | |
| Llama 4 MaverickCategory=Zero-Shot2025.10 | 71 | 70 | 71 | 78 | 76 | |
| Pixtral LargeCategory=Zero-Shot2025.10 | 68 | 62 | 74 | 73 | 72 | |
| ImageGuardCategory=Fine-tuned2025.10 | 68 | — | — | — | 78 | |
| LlavaGuardCategory=Fine-tuned, Policy=UnsafeBench policy (UP)2025.10 | 66 | 79 | 56 | 78 | 74 | |
| LlavaGuardCategory=Fine-tuned, Policy=LlavaGuard policy (LP)2025.10 | 63 | 80 | 52 | 77 | 73 | |
| Category-based RoutingCategory=Agentic2025.10 | 63 | 78 | 53 | 76 | 73 | |
| Safe-CLIPCategory=Fine-tuned2025.10 | 59 | 45 | 86 | 54 | 53 | |
| Llama 4 ScoutCategory=Zero-Shot2025.10 | 56 | 79 | 43 | 74 | 69 | |
| Claude Sonnet 3.7Category=Zero-Shot2025.10 | 55 | 87 | 40 | 75 | 69 | |
| Claude Sonnet 4Category=Zero-Shot2025.10 | 54 | 79 | 41 | 73 | 68 | |
| Claude Sonnet 3.5 v2Category=Zero-Shot2025.10 | 54 | 87 | 35 | 74 | 68 | |
| ICM AssistantCategory=Fine-tuned2025.10 | 54 | 65 | 46 | 71 | 66 | |
| SafeVisionCategory=Fine-tuned, Note=Results from original paper2025.10 | 50 | — | — | — | — | |
| Claude Opus 4Category=Zero-Shot2025.10 | 35 | 88 | 22 | 69 | 57 |