LLM Agent Defense on AgentDojo Workspace
85Clean UtilityTask Shield
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Task ShieldModel=GPT-4o-mini2024.12 | 85 | 69.58 | 1.25 | |
| No DefenseModel=GPT-4o-mini2024.12 | 82.5 | 59.17 | 17.92 | |
| Repeat PromptModel=GPT-4o2024.12 | 80 | 60.42 | 14.58 | |
| DelimitingModel=GPT-4o-mini2024.12 | 72.5 | 64.58 | 12.92 | |
| Tool FilterModel=GPT-4o-mini2024.12 | 70 | 64.58 | 2.5 | |
| Repeat PromptModel=GPT-4o-mini2024.12 | 70 | 61.25 | 8.33 | |
| No DefenseModel=GPT-4o2024.12 | 62.5 | 24.17 | 40.42 | |
| DelimitingModel=GPT-4o2024.12 | 62.5 | 30.42 | 35 | |
| Task ShieldModel=GPT-4o2024.12 | 62.5 | 62.5 | 0.42 | |
| PI DetectorModel=GPT-4o-mini2024.12 | 60 | 27.5 | 12.92 | |
| Tool FilterModel=GPT-4o2024.12 | 55 | 51.67 | 4.17 | |
| PI DetectorModel=GPT-4o2024.12 | 52.5 | 15.83 | 15 |