LLM Agent Defense on AgentDojo Slack
80.95Clean UtilityNo Defense
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| No DefenseModel=GPT-4o2024.12 | 80.95 | 63.81 | 92.38 | |
| Tool FilterModel=GPT-4o2024.12 | 80.95 | 47.62 | 8.57 | |
| Repeat PromptModel=GPT-4o2024.12 | 80.95 | 62.86 | 60 | |
| DelimitingModel=GPT-4o2024.12 | 80.95 | 61.9 | 80 | |
| Task ShieldModel=GPT-4o2024.12 | 80.95 | 64.76 | 0.95 | |
| Repeat PromptModel=GPT-4o-mini2024.12 | 71.43 | 33.33 | 13.33 | |
| DelimitingModel=GPT-4o-mini2024.12 | 71.43 | 56.19 | 48.57 | |
| No DefenseModel=GPT-4o-mini2024.12 | 66.67 | 48.57 | 57.14 | |
| Task ShieldModel=GPT-4o-mini2024.12 | 66.67 | 50.48 | 0.95 | |
| Tool FilterModel=GPT-4o-mini2024.12 | 57.14 | 45.71 | 7.62 | |
| PI DetectorModel=GPT-4o2024.12 | 28.57 | 25.71 | 12.38 | |
| PI DetectorModel=GPT-4o-mini2024.12 | 23.81 | 15.24 | 7.62 |