LLM Agent Defense on AgentDojo Overall
84.54Clean UtilityRepeat Prompt
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Repeat PromptModel=GPT-4o2024.12 | 84.54 | 67.25 | 27.82 | |
| Task ShieldModel=GPT-4o2024.12 | 73.2 | 69.79 | 2.07 | |
| Tool FilterModel=GPT-4o2024.12 | 72.16 | 56.28 | 6.84 | |
| DelimitingModel=GPT-4o2024.12 | 72.16 | 55.64 | 41.65 | |
| No DefenseModel=GPT-4o2024.12 | 69.07 | 50.08 | 47.69 | |
| No DefenseModel=GPT-4o-mini2024.12 | 68.04 | 49.92 | 27.19 | |
| Task ShieldModel=GPT-4o-mini2024.12 | 68.04 | 54.53 | 2.23 | |
| Repeat PromptModel=GPT-4o-mini2024.12 | 65.98 | 51.03 | 9.38 | |
| DelimitingModel=GPT-4o-mini2024.12 | 64.95 | 53.74 | 22.26 | |
| Tool FilterModel=GPT-4o-mini2024.12 | 61.86 | 55.17 | 4.93 | |
| PI DetectorModel=GPT-4o2024.12 | 41.24 | 21.14 | 7.95 | |
| PI DetectorModel=GPT-4o-mini2024.12 | 41.24 | 23.05 | 8.59 |