Prompt Injection Defense on AgentDojo R1–R3
1.76Attack Success Rate (ASR)RETA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| RETAModel=Llama-3.1-8B-Instruct, Attack=Static2026.06 | 1.76 | 36.3 | — | |
| PIGuardModel=Llama-3.1-8B-Instruct, Attack=Static2026.06 | 2.48 | 29.5 | — | |
| MetaSecAlignModel=Llama-3.1-8B-Instruct, Attack=Static2026.06 | 2.67 | 35.1 | — | |
| RETAModel=Qwen3-4B-Instruct-2507, Attack=Static2026.06 | 2.9 | 59.2 | — | |
| RETAModel=Qwen3-4B-Instruct-2507, Attack=Adaptive2026.06 | 2.92 | 55.8 | — | |
| PIGuardModel=Qwen3-4B-Instruct-2507, Attack=Static2026.06 | 3.33 | 33.8 | — | |
| MetaSecAlignModel=Qwen3-4B-Instruct-2507, Attack=Static2026.06 | 3.52 | 58.5 | — | |
| RETAModel=Llama-3.1-8B-Instruct, Attack=Adaptive2026.06 | 3.75 | 49.2 | — | |
| MELONModel=Qwen3-4B-Instruct-2507, Attack=Static2026.06 | 3.86 | 37.7 | — | |
| MELONModel=Llama-3.1-8B-Instruct, Attack=Static2026.06 | 4.52 | 28.4 | — | |
| PromptGuardModel=Llama-3.1-8B-Instruct, Attack=Static2026.06 | 4.57 | 18.1 | — | |
| MetaSecAlignModel=Llama-3.1-8B-Instruct, Attack=Adaptive2026.06 | 5 | 47.9 | — | |
| SandwichingModel=Llama-3.1-8B-Instruct, Attack=Static2026.06 | 6.57 | 34.6 | — | |
| SandwichingModel=Llama-3.1-8B-Instruct, Attack=Adaptive2026.06 | 7.08 | 40.4 | — | |
| PromptGuardModel=Qwen3-4B-Instruct-2507, Attack=Static2026.06 | 10.3 | 25.5 | — | |
| SpotlightingModel=Llama-3.1-8B-Instruct, Attack=Static2026.06 | 11 | 28.9 | — | |
| DataSentinelModel=Llama-3.1-8B-Instruct, Attack=Static2026.06 | 11.9 | 31.2 | — | |
| MetaSecAlignModel=Qwen3-4B-Instruct-2507, Attack=Adaptive2026.06 | 12.5 | 69.2 | — | |
| SpotlightingModel=Llama-3.1-8B-Instruct, Attack=Adaptive2026.06 | 12.5 | 39.6 | — | |
| NoneModel=Llama-3.1-8B-Instruct, Attack=Static2026.06 | 13.5 | 33 | — | |
| PromptGuardModel=Llama-3.1-8B-Instruct, Attack=Adaptive2026.06 | 13.8 | 34.2 | — | |
| NoneModel=Llama-3.1-8B-Instruct, Attack=Adaptive2026.06 | 14.2 | 47.1 | — | |
| MELONModel=Llama-3.1-8B-Instruct, Attack=Adaptive2026.06 | 14.6 | 40.8 | — | |
| SandwichingModel=Qwen3-4B-Instruct-2507, Attack=Adaptive2026.06 | 16.3 | 65 | — | |
| DataSentinelModel=Llama-3.1-8B-Instruct, Attack=Adaptive2026.06 | 16.7 | 38.8 | — | |
| SandwichingModel=Qwen3-4B-Instruct-2507, Attack=Static2026.06 | 17.1 | 49.1 | — | |
| PIGuardModel=Llama-3.1-8B-Instruct, Attack=Adaptive2026.06 | 18.8 | 40 | — | |
| MELONModel=Qwen3-4B-Instruct-2507, Attack=Adaptive2026.06 | 20.8 | 59.6 | — | |
| DataSentinelModel=Qwen3-4B-Instruct-2507, Attack=Static2026.06 | 23.4 | 36.1 | — | |
| PromptGuardModel=Qwen3-4B-Instruct-2507, Attack=Adaptive2026.06 | 27.9 | 56.7 | — | |
| DataSentinelModel=Qwen3-4B-Instruct-2507, Attack=Adaptive2026.06 | 27.9 | 66.3 | — | |
| SpotlightingModel=Qwen3-4B-Instruct-2507, Attack=Adaptive2026.06 | 28.3 | 62.1 | — | |
| SpotlightingModel=Qwen3-4B-Instruct-2507, Attack=Static2026.06 | 29.1 | 43.3 | — | |
| NoneModel=Qwen3-4B-Instruct-2507, Attack=Adaptive2026.06 | 32.1 | 57.5 | — | |
| NoneModel=Qwen3-4B-Instruct-2507, Attack=Static2026.06 | 33.3 | 44.5 | — | |
| PIGuardModel=Qwen3-4B-Instruct-2507, Attack=Adaptive2026.06 | 33.8 | 65.4 | — | |
| DataSentinelModel=Qwen3-4B-Instruct-2507, Attack=None2026.06 | — | — | 57.7 | |
| DataSentinelModel=Llama-3.1-8B-Instruct, Attack=None2026.06 | — | — | 44.3 | |
| MELONModel=Qwen3-4B-Instruct-2507, Attack=None2026.06 | — | — | 54.6 | |
| MELONModel=Llama-3.1-8B-Instruct, Attack=None2026.06 | — | — | 43.3 | |
| MetaSecAlignModel=Qwen3-4B-Instruct-2507, Attack=None2026.06 | — | — | 64.9 | |
| MetaSecAlignModel=Llama-3.1-8B-Instruct, Attack=None2026.06 | — | — | 46.4 | |
| NoneModel=Qwen3-4B-Instruct-2507, Attack=None2026.06 | — | — | 61.9 | |
| NoneModel=Llama-3.1-8B-Instruct, Attack=None2026.06 | — | — | 48.5 | |
| PIGuardModel=Qwen3-4B-Instruct-2507, Attack=None2026.06 | — | — | 63.9 | |
| PIGuardModel=Llama-3.1-8B-Instruct, Attack=None2026.06 | — | — | 45.4 | |
| PromptGuardModel=Qwen3-4B-Instruct-2507, Attack=None2026.06 | — | — | 63.9 | |
| PromptGuardModel=Llama-3.1-8B-Instruct, Attack=None2026.06 | — | — | 42.3 | |
| RETAModel=Qwen3-4B-Instruct-2507, Attack=None2026.06 | — | — | 60.8 | |
| RETAModel=Llama-3.1-8B-Instruct, Attack=None2026.06 | — | — | 45.4 | |
| SandwichingModel=Qwen3-4B-Instruct-2507, Attack=None2026.06 | — | — | 60.8 | |
| SandwichingModel=Llama-3.1-8B-Instruct, Attack=None2026.06 | — | — | 46.4 | |
| SpotlightingModel=Qwen3-4B-Instruct-2507, Attack=None2026.06 | — | — | 57.7 | |
| SpotlightingModel=Llama-3.1-8B-Instruct, Attack=None2026.06 | — | — | 44.3 |