Safety evaluation on UnsafeBench
89F1 ScoreBLPO
Evaluation results
| Method | Links | ||
|---|---|---|---|
| BLPOJudge Model=Llama-4-Maverick-17B-128E-instruct2026.02 | 89 | 90 | |
| GuardReasoner-VLSetting=Pixel-only2026.06 | 84.17 | — | |
| BLPOJudge Model=Llama-4-Scout-17B-16E-instruct2026.02 | 83 | 84 | |
| OPROJudge Model=Llama-4-Scout-17B-16E-instruct2026.02 | 81 | 83 | |
| SingGuard-4BSetting=Pixel-only2026.06 | 80.02 | — | |
| BLPOJudge Model=Qwen2.5-VL-32B-instruct2026.02 | 80 | 82 | |
| SingGuard-8BSetting=Pixel-only2026.06 | 78.84 | — | |
| TextGradJudge Model=Qwen2.5-VL-32B-instruct2026.02 | 78 | 81 | |
| SingGuard-2BSetting=Pixel-only2026.06 | 77.82 | — | |
| Qwen3-VL-235BSize=235B2025.12 | 74.4 | 80.9 | |
| Qwen2.5-VL-72BSize=72B2025.12 | 73.2 | 77.7 | |
| OMNIGUARD-3BSize=3B2025.12 | 72.3 | 81.1 | |
| OMNIGUARD-7BSize=7B2025.12 | 72.2 | 81.1 | |
| Qwen3-VL-235BSetting=Pixel-only2026.06 | 72.05 | — | |
| OPROJudge Model=Llama-4-Maverick-17B-128E-instruct2026.02 | 72 | 76 | |
| TextGradJudge Model=Llama-4-Scout-17B-16E-instruct2026.02 | 71 | 75 | |
| APO-imageJudge Model=Qwen2.5-VL-32B-instruct2026.02 | 70 | 76 | |
| APO-imageJudge Model=Llama-4-Scout-17B-16E-instruct2026.02 | 70 | 71 | |
| SafeGuard-VL-RLSetting=Pixel-only2026.06 | 69.06 | — | |
| OPROJudge Model=Qwen2.5-VL-32B-instruct2026.02 | 68 | 70 | |
| TextGradJudge Model=Llama-4-Maverick-17B-128E-instruct2026.02 | 68 | 73 | |
| APO-imageJudge Model=Llama-4-Maverick-17B-128E-instruct2026.02 | 67 | 70 | |
| No Optim.Judge Model=Qwen2.5-VL-32B-instruct2026.02 | 65 | 67 | |
| No Optim.Judge Model=Llama-4-Maverick-17B-128E-instruct2026.02 | 65 | 69 | |
| LlavaGuard-v1.2-7BSize=7B2025.12 | 63.4 | 77 | |
| No Optim.Judge Model=Llama-4-Scout-17B-16E-instruct2026.02 | 63 | 65 | |
| LLaVAShieldSetting=Pixel-only2026.06 | 60.87 | — | |
| LlavaGuardSetting=Pixel-only2026.06 | 57.79 | — | |
| Qwen2.5-VL-7BSize=7B2025.12 | 55.7 | 40.1 | |
| GPT-4oSize=-2025.12 | 55.2 | 74.7 | |
| GPT-5.1Setting=Pixel-only2026.06 | 54.27 | — | |
| ShieldGemma-2Setting=Pixel-only2026.06 | 53.19 | — | |
| Gemini3-ProSetting=Pixel-only2026.06 | 49.6 | — | |
| LlamaGuard4Setting=Pixel-only2026.06 | 48.52 | — | |
| Qwen3-VL-8BSetting=Pixel-only2026.06 | 48 | — | |
| Qwen2.5-Omni-7BSize=7B2025.12 | 47.3 | 67.8 | |
| LlamaGuard3-VisionSetting=Pixel-only2026.06 | 43.23 | — | |
| Qwen3-VL-4BSetting=Pixel-only2026.06 | 43.11 | — | |
| LLaMA Guard 3VSize=11B2025.12 | 0 | 61.9 | |
| Claude 3 Opus2026.07 | — | 73.39 | |
| Gemini 1.5 Pro2026.07 | — | 76.04 | |
| GPT-4o2026.07 | — | 76.34 | |
| InternVL2-8B2026.07 | — | 66.42 | |
| InternVL2.5-8B2026.07 | — | 76.29 | |
| MiShield-8B2026.07 | — | 77.86 | |
| Qwen2.5-9B2026.07 | — | 68.88 |