Safety Risk Detection on internal Agentic AI workflow benchmark
100PrecisionShieldGemma-9B
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| ShieldGemma-9BReasoning=Without2025.12 | 100 | 3 | 5 | 0 | |
| Qwen3Guard-8B-looseReasoning=Without2025.12 | 100 | 5 | 10 | 0 | |
| IBM-Granite-Guardian-3.3-8BReasoning=Without2025.12 | 98 | 28 | 43 | 1 | |
| AprielGuard-8BReasoning=Without2025.12 | 98 | 77 | 86 | 2 | |
| AprielGuard-8BReasoning=With2025.12 | 98 | 67 | 80 | 2 | |
| IBM-Granite-Guardian-3.3-8BReasoning=With2025.12 | 97 | 23 | 37 | 1 | |
| gpt-oss-safeguard-20BReasoning=With2025.12 | 97 | 46 | 62 | 2 | |
| Qwen3Guard-8B-strictReasoning=Without2025.12 | 96 | 25 | 40 | 2 | |
| IBM-Granite-Guardian-3.1-2BReasoning=Without2025.12 | 95 | 15 | 26 | 1 | |
| Llama-guard-3-8BReasoning=Without2025.12 | 86 | 23 | 36 | 5 | |
| Llama-guard-4-12BReasoning=Without2025.12 | 79 | 21 | 33 | 32 | |
| IBM-Granite-Guardian-3.2-3BReasoning=Without2025.12 | 65 | 84 | 73 | 64 | |
| IBM-Granite-Guardian-3.2-5BReasoning=Without2025.12 | 61 | 86 | 71 | 83 | |
| Llama-Prompt-Guard-2-86MReasoning=Without2025.12 | 60 | 75 | 67 | 71 | |
| Llama-guard-3-8BReasoning=Without2025.12 | 0.94 | 0.68 | 0.79 | 0.05 | |
| Llama-guard-4-12BReasoning=Without2025.12 | 0.92 | 0.67 | 0.77 | 0.06 | |
| Llama-guard-2-8BReasoning=Without2025.12 | 0.91 | 0.71 | 0.8 | 0.07 | |
| Qwen3Guard-8B-looseReasoning=Without2025.12 | 0.91 | 0.87 | 0.89 | 0.09 | |
| ShieldGemma-9BReasoning=Without2025.12 | 0.88 | 0.52 | 0.65 | 0.07 | |
| IBM-Granite-Guardian-3.2-5BReasoning=Without2025.12 | 0.87 | 0.86 | 0.86 | 0.13 | |
| AprielGuard-8BReasoning=Without2025.12 | 0.87 | 0.89 | 0.88 | 0.11 | |
| AprielGuard-8BReasoning=With2025.12 | 0.87 | 0.87 | 0.87 | 0.12 | |
| IBM-Granite-Guardian-3.3-8BReasoning=Without2025.12 | 0.85 | 0.9 | 0.87 | 0.15 | |
| gpt-oss-safeguard-20BReasoning=With2025.12 | 0.85 | 0.86 | 0.85 | 0.14 | |
| IBM-Granite-Guardian-3.3-8BReasoning=With2025.12 | 0.84 | 0.88 | 0.86 | 0.16 | |
| IBM-Granite-Guardian-3.1-2BReasoning=Without2025.12 | 0.83 | 0.86 | 0.84 | 0.17 | |
| Qwen3Guard-8B-strictReasoning=Without2025.12 | 0.83 | 0.94 | 0.88 | 0.18 | |
| IBM-Granite-Guardian-3.2-3BReasoning=Without2025.12 | 0.8 | 0.86 | 0.83 | 0.21 | |
| Llama-Prompt-Guard-2-86MReasoning=Without2025.12 | 0.48 | 0.02 | 0.04 | 0.02 |