Content Moderation on SafeWatch-Real (val)
95.2Sexual AccuracyGemma2-9B
Evaluation Results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Gemma2-9BModel Category=Open-source2026.05 | 95.2 | 38.1 | 75.7 | 26.2 | 68.9 | 52.9 | 83.5 | 63 | 62.7 | |
| OmniGuard-7BModel Category=Guardrails2026.05 | 94.1 | 17.7 | 23.8 | 25.8 | 41.1 | 40.4 | 75.2 | 45.4 | 45.2 | |
| GPT-4oModel Category=Closed-source2026.05 | 92 | 35.4 | 82.7 | 27.9 | 80 | 52.5 | 84.6 | 65 | 64.6 | |
| Qwen2-72BModel Category=Open-source2026.05 | 90.8 | 22.5 | 75.3 | 16.9 | 63.3 | 14.1 | 82.9 | 52.2 | 49.6 | |
| Qwen2-VL-2B-ftModel Category=Open-source2026.05 | 88.6 | 48.3 | 80.9 | 90.1 | 57.8 | 91.4 | 54.1 | 73 | 73.5 | |
| QwenGuard-7BModel Category=Guardrails2026.05 | 88.2 | 8.2 | 4.8 | 0 | 64.5 | 1.2 | 66.2 | 33.3 | 25.1 | |
| SAFELENSModel Category=Ours2026.05 | 87.9 | 78.9 | 83.1 | 78.2 | 74.4 | 93.3 | 84.6 | 82.9 | 81.7 | |
| SAFELENS-S1Model Category=Ours2026.05 | 87.2 | 77.6 | 83.1 | 76.5 | 73.3 | 92.9 | 83.5 | 82 | 80.8 | |
| SAFELENS-S2Model Category=Ours2026.05 | 87 | 78.9 | 80.9 | 85.7 | 74.4 | 92.6 | 73.6 | 81.9 | 80 | |
| OmniGuard-3B-ftModel Category=Guardrails2026.05 | 85.3 | 64 | 79.2 | 93.5 | 73.3 | 90.6 | 45.4 | 75.9 | 73.5 | |
| Gemini-1.5-ProModel Category=Closed-source2026.05 | 84.9 | 37.4 | 87.4 | 26.9 | 75.6 | 61.2 | 88.2 | 65.9 | 66.4 | |
| OmniGuard-3BModel Category=Guardrails2026.05 | 82.5 | 22.4 | 26 | 53.4 | 37.8 | 47.1 | 61.8 | 47.3 | 47.7 | |
| Qwen2-VL-2BModel Category=Open-source2026.05 | 79.9 | 4.8 | 59.3 | 3.7 | 2.2 | 0.8 | 94.9 | 35.1 | 30.6 | |
| SafeWatch-8BModel Category=Guardrails2026.05 | 76.1 | 29.9 | 77 | 50.3 | 35.5 | 64.3 | 93.5 | 60.9 | 64.4 |