Agent defense evaluation on AgentDojo
69.15Utility under AttackSanitizer
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| SanitizerBase LLM=GPT-4o, Category=Simplified Firewalls2025.10 | 69.15 | 74.09 | 0 | |
| PromptArmor*Base LLM=GPT-4o2025.10 | 68.68 | — | 0.47 | |
| Repeat promptBase LLM=GPT-4o2025.10 | 67.25 | 85.53 | 27.82 | |
| CombinedBase LLM=GPT-4o, Category=Simplified Firewalls2025.10 | 58.59 | 58.41 | 0 | |
| ToolfilterBase LLM=GPT-4o2025.10 | 56.28 | 73.13 | 6.84 | |
| SpotlightingBase LLM=GPT-4o2025.10 | 55.64 | 71.66 | 41.65 | |
| CaMeL*Base LLM=GPT-4o2025.10 | 54.5 | 53.6 | 0 | |
| Melon-Aug*Base LLM=GPT-4o2025.10 | 52.46 | 76.29 | 1.27 | |
| MinimizerBase LLM=GPT-4o, Category=Simplified Firewalls2025.10 | 50.03 | 70.01 | 18.15 | |
| NoneBase LLM=GPT-4o2025.10 | 50.01 | 83.02 | 57.69 | |
| Melon*Base LLM=GPT-4o2025.10 | 32.91 | 68.04 | 0.95 | |
| Pi DetectorBase LLM=GPT-4o2025.10 | 21.14 | 41.49 | 7.95 |