Indirect Prompt Injection on Amazon Reviews
99.8ASRObliInjection
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ObliInjectionTarget LLM=Qwen-2.5-7B2025.12 | 99.8 | — | |
| ObliInjectionTarget LLM=Llama4-17B2025.12 | 99.8 | — | |
| ObliInjectionTarget LLM=Llama-3-8B2025.12 | 99.4 | — | |
| ObliInjectionPrevention=Leave-one-segment-out, Aggregation=Averaged across seven LLMs2025.12 | 99.3 | — | |
| JudgeDeceiverTarget LLM=Qwen-2.5-7B2025.12 | 99.2 | — | |
| ObliInjection-CETarget LLM=Llama-3-8B2025.12 | 99.2 | — | |
| ObliInjectionTarget LLM=Mistral-7B2025.12 | 99.2 | — | |
| ObliInjectionTarget LLM=Qwen3-4B2025.12 | 98.8 | — | |
| ObliInjectionTarget LLM=Falcon3-7B2025.12 | 98.2 | — | |
| ObliInjectionTarget LLM=Llama-3.1-8B2025.12 | 98 | — | |
| ObliInjection-CETarget LLM=Falcon3-7B2025.12 | 96.8 | — | |
| ObliInjection-CETarget LLM=Llama4-17B2025.12 | 96.6 | — | |
| ObliInjectionPrevention=Segment Delimiters, Aggregation=Averaged across seven LLMs2025.12 | 96.3 | — | |
| ObliInjection-CETarget LLM=Llama-3.1-8B2025.12 | 96 | — | |
| ObliInjection-CETarget LLM=Qwen3-4B2025.12 | 96 | — | |
| ObliInjection-CETarget LLM=Qwen-2.5-7B2025.12 | 86.8 | — | |
| ObliInjection-GCGTarget LLM=Llama4-17B2025.12 | 85.4 | — | |
| JudgeDeceiverTarget LLM=Llama-3.1-8B2025.12 | 78.4 | — | |
| ObliInjectionPrevention=Llama-3-8B-StruQ2025.12 | 77.9 | — | |
| JudgeDeceiverTarget LLM=Mistral-7B2025.12 | 75.8 | — | |
| ObliInjection-CETarget LLM=Mistral-7B2025.12 | 64.2 | — | |
| ObliInjectionPrevention=Llama-3-8B-SecAlign2025.12 | 63.8 | — | |
| JudgeDeceiverTarget LLM=Llama-3-8B2025.12 | 62.2 | — | |
| ObliInjection-GCGTarget LLM=Llama-3-8B2025.12 | 61.6 | — | |
| ObliInjectionPrevention=Llama-3-8B-SecAlign-Adapt, Note=Fine-tuned with DPO for 3 epochs using attack samples2025.12 | 53.3 | — | |
| JudgeDeceiverTarget LLM=Qwen3-4B2025.12 | 49.2 | — | |
| Neural ExecTarget LLM=Llama-3-8B2025.12 | 41.2 | — | |
| JudgeDeceiverTarget LLM=Falcon3-7B2025.12 | 32.6 | — | |
| ObliInjection-GCGTarget LLM=Qwen3-4B2025.12 | 32.4 | — | |
| Combined AttackTarget LLM=Qwen-2.5-7B2025.12 | 16.4 | — | |
| Combined AttackTarget LLM=Qwen3-4B2025.12 | 15.6 | — | |
| ObliInjection-GCGTarget LLM=Llama-3.1-8B2025.12 | 15.2 | — | |
| Combined AttackTarget LLM=Llama4-17B2025.12 | 9.4 | — | |
| Neural ExecTarget LLM=Llama4-17B2025.12 | 7 | — | |
| Neural ExecTarget LLM=Qwen3-4B2025.12 | 1.2 | — | |
| ObliInjection-GCGTarget LLM=Qwen-2.5-7B2025.12 | 1.2 | — | |
| Neural ExecTarget LLM=Falcon3-7B2025.12 | 0.4 | — | |
| JudgeDeceiverTarget LLM=Llama4-17B2025.12 | 0.2 | — | |
| Combined AttackTarget LLM=Llama-3-8B2025.12 | 0 | — | |
| Combined AttackTarget LLM=Llama-3.1-8B2025.12 | 0 | — | |
| Combined AttackTarget LLM=Mistral-7B2025.12 | 0 | — | |
| Combined AttackTarget LLM=Falcon3-7B2025.12 | 0 | — | |
| Neural ExecTarget LLM=Llama-3.1-8B2025.12 | 0 | — | |
| Neural ExecTarget LLM=Mistral-7B2025.12 | 0 | — | |
| Neural ExecTarget LLM=Qwen-2.5-7B2025.12 | 0 | — | |
| ObliInjection-GCGTarget LLM=Mistral-7B2025.12 | 0 | — | |
| ObliInjection-GCGTarget LLM=Falcon3-7B2025.12 | 0 | — | |
| Combined AttackTarget LLM=Average2025.12 | — | 5.9 | |
| JudgeDeceiverTarget LLM=Average2025.12 | — | 56.8 | |
| Neural ExecTarget LLM=Average2025.12 | — | 7.1 | |
| ObliInjectionTarget LLM=Average2025.12 | — | 99 | |
| ObliInjection-CETarget LLM=Average2025.12 | — | 90.8 | |
| ObliInjection-GCGTarget LLM=Average2025.12 | — | 28 |