Prompt Injection Defense on AgentDojo
89.99Benign UtilityPIM
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| PIMModel=Gemma-4-31B-IT2026.06 | 89.99 | — | — | — | — | — | — | — | — | — | 0 | |
| SOMModel=Gemma-4-31B-IT2026.06 | 89.99 | — | — | — | — | — | — | — | — | — | 0 | |
| BaselineModel=Gemma-4-31B-IT2026.06 | 89.46 | — | — | — | — | — | — | — | — | — | 3.69 | |
| PIMModel=Claude Sonnet 4.62026.06 | 87.25 | — | — | — | — | — | — | — | — | — | 0 | |
| SOMModel=Claude Sonnet 4.62026.06 | 86.72 | — | — | — | — | — | — | — | — | — | 0 | |
| BaselineModel=Claude Sonnet 4.62026.06 | 86.51 | — | — | — | — | — | — | — | — | — | 0 | |
| BaselineModel=GPT-5.32026.06 | 85.25 | — | — | — | — | — | — | — | — | — | 0.53 | |
| PIMModel=GPT-5.32026.06 | 84.3 | — | — | — | — | — | — | — | — | — | 0 | |
| SOMModel=GPT-5.32026.06 | 84.19 | — | — | — | — | — | — | — | — | — | 0.42 | |
| VanillaModel=Qwen3-32B2025.11 | 77.3 | — | 51.5 | 46.8 | 50.3 | 48.2 | 69.3 | 62.4 | 46.4 | 72.3 | — | |
| SEP DefenseModel=Qwen3-32B2025.11 | 77.3 | — | 53.5 | 45.2 | 55.7 | 48.9 | 50.3 | 54.2 | 47.6 | 68.9 | — | |
| PromptArmorModel=Qwen3-32B2025.11 | 77.3 | 0 | 60.7 | 25.9 | 59.3 | 30.4 | 62.3 | 27.8 | 43.2 | 65.9 | — | |
| IntentGuardModel=Qwen3-32B2025.11 | 77.3 | 0 | 71.6 | 4.3 | 72.3 | 3.9 | 71 | 3.9 | 68.7 | 9.2 | — | |
| DataFilterBackend LLM=GPT-4o, Attack Type=important message2026.04 | 68 | — | — | — | — | — | — | — | — | — | 8 | |
| VanillaModel=gpt-oss-20B2025.11 | 64.9 | — | 45.2 | 49.1 | 46.3 | 50.7 | 58.2 | 64.2 | 38.9 | 73.1 | — | |
| SEP DefenseModel=gpt-oss-20B2025.11 | 64.9 | — | 47.9 | 47.4 | 49.3 | 49.6 | 47.1 | 55.1 | 40.2 | 69.5 | — | |
| PromptArmorModel=gpt-oss-20B2025.11 | 64.9 | 0 | 53.8 | 27.8 | 52.1 | 31.7 | 55.4 | 30.2 | 40.1 | 67.1 | — | |
| IntentGuardModel=gpt-oss-20B2025.11 | 64.9 | 0 | 67.3 | 5.1 | 67.8 | 4.7 | 66.4 | 4.9 | 64.3 | 10.4 | — | |
| PISanitizerBackend LLM=GPT-4o, Attack Type=important message2026.04 | 63 | — | — | — | — | — | — | — | — | — | 7 | |
| No DefenseBackend LLM=GPT-4o, Attack Type=important message2026.04 | 52 | — | — | — | — | — | — | — | — | — | 42 | |
| PromptArmorBackend LLM=GPT-4o, Attack Type=important message2026.04 | 38 | — | — | — | — | — | — | — | — | — | 6 | |
| SecAlign++Backend LLM=GPT-4o, Attack Type=important message2026.04 | 19 | — | — | — | — | — | — | — | — | — | 0 | |
| Attn.TrackerBackend LLM=GPT-4o, Attack Type=important message2026.04 | — | — | — | — | — | — | — | — | — | — | 6 | |
| DataSentinelBackend LLM=GPT-4o, Attack Type=important message2026.04 | — | — | — | — | — | — | — | — | — | — | 32 | |
| PIGuardBackend LLM=GPT-4o, Attack Type=important message2026.04 | — | — | — | — | — | — | — | — | — | — | 15 | |
| PromptGuardBackend LLM=GPT-4o, Attack Type=important message2026.04 | — | — | — | — | — | — | — | — | — | — | 22 |