Prompt Classification on SEA-SafeguardBench
93.6AUPRC (Average)SEA-Guard-12B
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| SEA-Guard-12Bevaluation_mode=zero-shot2026.02 | 93.6 | — | — | — | — | |
| SEA-Guard-8Bevaluation_mode=zero-shot2026.02 | 93 | — | — | — | — | |
| SEA-Guard-4Bevaluation_mode=zero-shot2026.02 | 92.6 | — | — | — | — | |
| Qwen3Guard-Gen 8Bevaluation_mode=zero-shot2026.02 | 91 | — | — | — | — | |
| LlamaGuard-3 8Bevaluation_mode=zero-shot2026.02 | 90.4 | — | — | — | — | |
| Qwen3Guard-Gen 4Bevaluation_mode=zero-shot2026.02 | 90 | — | — | — | — | |
| PolyGuard-Ministral 8Bevaluation_mode=zero-shot2026.02 | 88.3 | — | — | — | — | |
| ShieldGemma 9Bevaluation_mode=zero-shot2026.02 | 82.8 | — | — | — | — | |
| ShieldGemma 27Bevaluation_mode=zero-shot2026.02 | 82.5 | — | — | — | — | |
| SEA-GuardModel Size=12B2026.02 | 80 | 99.5 | 99 | 59.7 | 61.7 | |
| SEA-GuardModel Size=8B2026.02 | 79.5 | 99.2 | 98.6 | 61.2 | 59 | |
| SEA-GuardModel Size=4B2026.02 | 79.4 | 99.3 | 98.8 | 58.3 | 61.2 | |
| ShieldGemmaModel Size=27B2026.02 | 78 | 98 | 96 | 58.7 | 59.4 | |
| Qwen3Guard-GenModel Size=4B2026.02 | 75.4 | 98.4 | 97.3 | 56.8 | 49 | |
| ShieldGemmaModel Size=9B2026.02 | 75.1 | 97.2 | 95.3 | 52.2 | 55.7 | |
| Qwen3Guard-GenModel Size=8B2026.02 | 74.6 | 98.7 | 98 | 54.2 | 47.6 | |
| LlamaGuardModel Version=3, Model Size=8B2026.02 | 73.1 | 97.4 | 95.6 | 55.4 | 44.1 | |
| ShieldGemmaModel Size=2B2026.02 | 72.8 | 95.8 | 90.6 | 53.2 | 51.8 | |
| PolyGuard-QwenModel Size=8B2026.02 | 72.1 | 98.6 | 94.9 | 53.8 | 41 | |
| PolyGuard-MinistralModel Size=8B2026.02 | 71.4 | 98.9 | 95.5 | 49.9 | 41.1 | |
| OpenAI Moderation2026.02 | 66.9 | 95.3 | 86.4 | 45.5 | 40.3 | |
| LionGuard-22026.02 | 65.7 | 95.8 | 78.5 | 46.7 | 41.9 | |
| X-Guard2026.02 | 65.2 | 97 | 86.1 | 42.5 | 35.1 | |
| LlamaGuardModel Version=3, Model Size=1B2026.02 | 64.4 | 91.8 | 86.4 | 45.7 | 33.9 | |
| LlamaGuardModel Version=4, Model Size=12B2026.02 | 64.4 | 94.6 | 84.7 | 46 | 32.4 | |
| PolyGuard-QwenModel Size=0.5B2026.02 | 63.3 | 97.5 | 82.6 | 40.8 | 32.4 | |
| Azure AI Content Safety2026.02 | 59.8 | 88.5 | 83.1 | 37.6 | 30.2 | |
| Google Model Armor2026.02 | 59 | 86.6 | 75.6 | 40.1 | 33.8 | |
| LakeraGuard2026.02 | 58.3 | 88.9 | 76.6 | 30 | 37.8 |