Indirect Prompt Injection Defense Evaluation on AgentDojo TOOLKNOWLEDGE attack suite
6.66Latency (s)PIGuard
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| PIGuardTarget Agent Model=Gemini-2.5 Flash2026.03 | 6.66 | 4,088 | 379 | |
| PI DetectorTarget Agent Model=Gemini-2.5 Flash2026.03 | 8.59 | 6,037 | 512 | |
| PromptGuardTarget Agent Model=Gemini-2.5 Flash2026.03 | 10.08 | 10,460 | 701 | |
| No DefenseTarget Agent Model=Gemini-2.5 Flash2026.03 | 11.48 | 11,924 | 811 | |
| PIGuardTarget Agent Model=Llama3.3-70B2026.03 | 11.76 | 23,598 | 385 | |
| SpotlightingTarget Agent Model=Gemini-2.5 Flash2026.03 | 12.56 | 14,046 | 940 | |
| PromptGuardTarget Agent Model=Llama3.3-70B2026.03 | 13.04 | 23,807 | 424 | |
| PI DetectorTarget Agent Model=Llama3.3-70B2026.03 | 13.59 | 24,172 | 449 | |
| SandwichingTarget Agent Model=Gemini-2.5 Flash2026.03 | 14.06 | 15,031 | 952 | |
| No DefenseTarget Agent Model=Llama3.3-70B2026.03 | 14.23 | 22,770 | 467 | |
| SpotlightingTarget Agent Model=Llama3.3-70B2026.03 | 14.73 | 23,926 | 469 | |
| MELONTarget Agent Model=Gemini-2.5 Flash2026.03 | 16.55 | 11,187 | 993 | |
| DataSentinelTarget Agent Model=Llama3.3-70B2026.03 | 17.44 | 22,022 | 458 | |
| PromptArmorTarget Agent Model=Llama3.3-70B2026.03 | 17.68 | 19,013 | 371 | |
| PromptArmorTarget Agent Model=Gemini-2.5 Flash2026.03 | 21.05 | 6,441 | 496 | |
| SandwichingTarget Agent Model=Llama3.3-70B2026.03 | 24.2 | 40,738 | 794 | |
| DataSentinelTarget Agent Model=Gemini-2.5 Flash2026.03 | 25.82 | 7,715 | 680 | |
| MELONTarget Agent Model=Llama3.3-70B2026.03 | 29.45 | 41,800 | 1,039 | |
| AttriGuardTarget Agent Model=Gemini-2.5 Flash2026.03 | 32.4 | 12,804 | 771 | |
| AttriGuardTarget Agent Model=Llama3.3-70B2026.03 | 41.22 | 45,419 | 965 | |
| IPIGuardTarget Agent Model=Llama3.3-70B2026.03 | 51.51 | 74,029 | 1,480 | |
| IPIGuardTarget Agent Model=Gemini-2.5 Flash2026.03 | 102.91 | 43,862 | 12,138 | |
| CaMelTarget Agent Model=Llama3.3-70B2026.03 | 105.44 | 89,720 | 3,242 | |
| CaMelTarget Agent Model=Gemini-2.5 Flash2026.03 | 165.47 | 126,572 | 22,727 |