Trajectory-level safety evaluation on ATBench (test)
0.928AccuracyAgentDoG-Qwen3-4B
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| AgentDoG-Qwen3-4BModel Category=Our Models2026.01 | 0.928 | 0.905 | 0.956 | 0.93 | |
| GPT-5.2Model Category=Closed-Source2026.01 | 0.9 | 0.847 | 0.976 | 0.907 | |
| AgentDoG-Llama3.1-8BModel Category=Our Models2026.01 | 0.876 | 0.809 | 0.984 | 0.888 | |
| AgentDoG-Qwen2.5-7BModel Category=Our Models2026.01 | 0.874 | 0.821 | 0.956 | 0.884 | |
| Gemini-3-ProModel Category=Closed-Source2026.01 | 0.872 | 0.927 | 0.808 | 0.863 | |
| Qwen3-235B-A22B-Instruct-2507Model Category=Open-Source2026.01 | 0.846 | 0.994 | 0.696 | 0.819 | |
| ShieldAgentModel Category=Guard Models2026.01 | 0.76 | 0.901 | 0.584 | 0.709 | |
| Gemini-3-FlashModel Category=Closed-Source2026.01 | 0.756 | 0.985 | 0.52 | 0.681 | |
| PolyGuardModel Category=Guard Models2026.01 | 0.738 | 0.687 | 0.876 | 0.77 | |
| QwQ-32BModel Category=Open-Source2026.01 | 0.63 | 0.933 | 0.28 | 0.431 | |
| Qwen3-4B-Instruct-2507Model Category=Open-Source2026.01 | 0.616 | 0.641 | 0.528 | 0.579 | |
| Qwen2.5-7B-InstructModel Category=Open-Source2026.01 | 0.594 | 0.658 | 0.392 | 0.491 | |
| LlamaGuard4-12BModel Category=Guard Models, Note=Shows high precision but low recall on ATBench2026.01 | 0.581 | 1 | 0.164 | 0.282 | |
| JoySafetyModel Category=Guard Models2026.01 | 0.569 | 0.626 | 0.348 | 0.447 | |
| Qwen3-GuardModel Category=Guard Models, Note=Shows high precision but low recall on ATBench2026.01 | 0.553 | 1 | 0.108 | 0.195 | |
| LlamaGuard3-8BModel Category=Guard Models, Note=Shows high precision but low recall on ATBench2026.01 | 0.533 | 1 | 0.068 | 0.127 | |
| ShieldGemma-9BModel Category=Guard Models2026.01 | 0.511 | 0.8 | 0.032 | 0.062 | |
| ShieldGemma-27BModel Category=Guard Models2026.01 | 0.509 | 0.778 | 0.028 | 0.054 | |
| NemoGuardModel Category=Guard Models2026.01 | 0.499 | 0.5 | 0.412 | 0.452 | |
| Llama3.1-8B-InstructModel Category=Open-Source2026.01 | 0.496 | 0.498 | 0.992 | 0.663 |