Agent Safety Auditing on ASSE-Security
97.6AccuracyTRACES
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| TRACESBackbone=Llama3.1-8B2026.05 | 97.6 | 98.2 | 96.6 | 62.1 | 96.7 | |
| TRACESBackbone=Qwen3-4B2026.05 | 95 | 95.6 | 95 | 56.3 | 96.9 | |
| AgentDoGBackbone=Llama3.1-8B2026.05 | 88.9 | 85.7 | 90.6 | 56.9 | 79.3 | |
| AgentDoGBackbone=Qwen3-4B2026.05 | 87.8 | 88.6 | 95.1 | 51.9 | 76.2 | |
| ShieldAgent2026.05 | 82.7 | 84.6 | 83.6 | 47.8 | 66.5 | |
| AgentAuditor2026.05 | 81 | 83 | 86.2 | 46.6 | 70.7 | |
| GPT-5.22026.05 | 73.8 | 69.1 | 58.5 | 47.6 | 74.9 | |
| Gemini-3-Flash2026.05 | 62.8 | 56.7 | 48.8 | 32.9 | 66.4 | |
| Llama3.1-8B-Instruct2026.05 | 53.1 | 38.5 | 25.6 | 9.8 | 56.9 | |
| PolyGuard2026.05 | 51.6 | 42.4 | 38.2 | 45.6 | 63.3 | |
| Qwen3-4B2026.05 | 50 | 66.7 | 100 | 75 | 50 | |
| LlamaGuard3-8B2026.05 | 47.6 | 12.2 | 7.3 | 7.3 | 49 | |
| JoySafety2026.05 | 39.2 | 32.7 | 29.8 | 30.6 | 55.2 |