Agent Safety Auditing on ASSE Strict
83.5AccuracyShieldAgent
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| ShieldAgent2026.05 | 83.5 | 86.2 | 82.4 | 14.6 | 57.8 | |
| TRACESBackbone=Qwen3-4B2026.05 | 82.1 | 87.9 | 92.6 | 34.9 | 77 | |
| TRACESBackbone=Llama3.1-8B2026.05 | 82.1 | 88.1 | 94.6 | 32.2 | 75.1 | |
| AgentDoGBackbone=Llama3.1-8B2026.05 | 81.3 | 82.2 | 86.7 | 24.8 | 59.6 | |
| AgentDoGBackbone=Qwen3-4B2026.05 | 81.1 | 84.2 | 89.2 | 18.9 | 55.7 | |
| AgentAuditor2026.05 | 76.6 | 79.1 | 75.4 | 12.1 | 54.3 | |
| GPT-5.22026.05 | 65.9 | 65.8 | 58.1 | 30.5 | 67.4 | |
| Gemini-3-Flash2026.05 | 62.8 | 47.6 | 56.1 | 26.8 | 66.7 | |
| Qwen3-4B2026.05 | 56.1 | 70.9 | 99 | 60.3 | 50 | |
| PolyGuard2026.05 | 54.4 | 53.7 | 52.8 | 10 | 55.1 | |
| Llama3.1-8B-Instruct2026.05 | 49.3 | 24.2 | 14.4 | 2.8 | 53.4 | |
| LlamaGuard3-8B2026.05 | 47.3 | 12.4 | 6.6 | 3.5 | 53.1 | |
| JoySafety2026.05 | 33.8 | 30.6 | 26.2 | 14.5 | 56.4 |