Agent Task Performance on AgentDojo Travel
13.57Attack Success RateNo Defense
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| No DefenseModel=GPT-4o-mini2024.12 | 13.57 | — | 55 | 47.14 | |
| No DefenseModel=GPT-4o, Attack Type=Important Instructions2024.12 | 11.43 | 64.29 | — | — | |
| No DefenseModel=GPT-4o2024.12 | 11.43 | — | 65 | 64.29 | |
| Repeat PromptModel=GPT-4o2024.12 | 7.14 | — | 90 | 72.14 | |
| DelimitingModel=GPT-4o-mini2024.12 | 7.14 | — | 60 | 52.14 | |
| Baseline2026.01 | 6.6 | 74 | — | 66.4 | |
| Tool FilterModel=GPT-4o2024.12 | 5.71 | — | 90 | 70 | |
| Tool filter2026.01 | 4.6 | 78 | — | 75.1 | |
| DelimitingModel=GPT-4o2024.12 | 3.57 | — | 75 | 72.14 | |
| PI detector2026.01 | 2.9 | 41 | — | 33.1 | |
| Repeat prompt2026.01 | 2 | 77 | — | 70.3 | |
| Delimiter2026.01 | 1.6 | 73 | — | 70 | |
| Task ShieldModel=GPT-4o, Attack Type=Important Instructions2024.12 | 1.43 | 72.86 | — | — | |
| Task ShieldModel=GPT-4o2024.12 | 1.43 | — | 80 | 72.86 | |
| Tool FilterModel=GPT-4o-mini2024.12 | 0.71 | — | 60 | 58.57 | |
| Task ShieldModel=GPT-4o-mini2024.12 | 0.71 | — | 55 | 49.29 | |
| Task ShieldModel=GPT-4o, Attack Type=Injecagent2024.12 | 0 | 67.86 | — | — | |
| No DefenseModel=GPT-4o, Attack Type=Injecagent2024.12 | 0 | 72.14 | — | — | |
| Task ShieldModel=GPT-4o, Attack Type=Ignore Previous2024.12 | 0 | 70.71 | — | — | |
| No DefenseModel=GPT-4o, Attack Type=Ignore Previous2024.12 | 0 | 77.14 | — | — | |
| PI DetectorModel=GPT-4o2024.12 | 0 | — | 30 | 16.43 | |
| Repeat PromptModel=GPT-4o-mini2024.12 | 0 | — | 70 | 54.29 | |
| PI DetectorModel=GPT-4o-mini2024.12 | 0 | — | 25 | 14.29 | |
| AGENTRIM2026.01 | 0 | 73 | — | 73.7 |