Visual Compliance Verification on LlavaGuard 1290 samples (test)
93Unsafe F1 ScoreCompAgent
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| CompAgentCategory=Proposed2025.10 | 93 | 90 | 96 | 93 | 93 | |
| LlavaGuardCategory=Fine-tuned, Policy=LlavaGuard policy (LP)2025.10 | 91 | 88 | 95 | 92 | 92 | |
| SafeVisionCategory=Fine-tuned, Note=Results from original paper2025.10 | 81 | — | — | — | — | |
| Claude Sonnet 3.5 v2Category=Zero-Shot2025.10 | 61 | 76 | 51 | 73 | 71 | |
| Category-based RoutingCategory=Agentic2025.10 | 61 | 72 | 53 | 72 | 69 | |
| ICM AssistantCategory=Fine-tuned2025.10 | 59 | 61 | 58 | 68 | 66 | |
| Pixtral LargeCategory=Zero-Shot2025.10 | 57 | 59 | 55 | 66 | 64 | |
| Llama 4 MaverickCategory=Zero-Shot2025.10 | 55 | 78 | 43 | 71 | 67 | |
| Claude Sonnet 3.7Category=Zero-Shot2025.10 | 52 | 77 | 39 | 70 | 65 | |
| Claude Sonnet 4Category=Zero-Shot2025.10 | 48 | 81 | 34 | 69 | 63 | |
| Llama 4 ScoutCategory=Zero-Shot2025.10 | 42 | 79 | 29 | 67 | 61 | |
| Safe-CLIPCategory=Fine-tuned2025.10 | 36 | 27 | 55 | 56 | 51 | |
| Claude Opus 4Category=Zero-Shot2025.10 | 30 | 76 | 19 | 64 | 53 |