Agentic Security Evaluation on AgentDojo v1 (97 benign tasks, 27 injection tasks)
60.8Utility ScoreNoDefense
Evaluation results
| Method | Links | ||
|---|---|---|---|
| NoDefenseModel=Qwen-plus2026.05 | 60.8 | 88.9 | |
| FIDESModel=Qwen-plus2026.05 | 59.8 | 85.2 | |
| NoDefenseModel=GPT-4o-mini2026.05 | 59.8 | 88.9 | |
| FIDESModel=GPT-4o-mini2026.05 | 55.7 | 88.9 | |
| FIDESModel=Qwen-max2026.05 | 50.5 | 96.3 | |
| FIDESModel=Qwen2.5-72B2026.05 | 48.5 | 96.3 | |
| NoDefenseModel=Qwen-max2026.05 | 46.4 | 96.3 | |
| NoDefenseModel=Qwen2.5-72B2026.05 | 46.4 | 96.3 | |
| PACTModel=GPT-4o-mini2026.05 | 46.4 | 100 | |
| PACTModel=Qwen-plus2026.05 | 45.4 | 100 | |
| PACTModel=Qwen2.5-72B2026.05 | 42.3 | 100 | |
| PACTModel=Qwen-max2026.05 | 38.1 | 100 | |
| CaMeLModel=GPT-4o-mini2026.05 | 30.9 | 100 | |
| CaMeLModel=Qwen-plus2026.05 | 29.9 | 100 | |
| CaMeLModel=Qwen-max2026.05 | 29.9 | 100 | |
| CaMeLModel=Qwen2.5-72B2026.05 | 28.9 | 100 | |
| NoDefenseModel=Qwen-turbo2026.05 | 20.6 | 92.6 | |
| FIDESModel=Qwen-turbo2026.05 | 19.6 | 88.9 | |
| PACTModel=Qwen-turbo2026.05 | 18.6 | 100 | |
| CaMeLModel=Qwen-turbo2026.05 | 11.3 | 100 |