Targeted Prompt Injection on ShipManager
100Attack Success RateTIP
Evaluation results
| Method | Links | |
|---|---|---|
| TIPDefense=No Defense, Victim Model=Qwen2.5-72B-Instruct2026.03 | 100 | |
| TIPDefense=No Defense, Victim Model=Llama3.3-70B-Instruct2026.03 | 100 | |
| TIPDefense=Perplexity Filtering, Victim Model=Llama3.3-70B-Instruct2026.03 | 100 | |
| TIPDefense=Finetuned Detector, Victim Model=Llama3.1-8B-Instruct2026.03 | 98 | |
| TIPDefense=Perplexity Filtering, Victim Model=Qwen2.5-72B-Instruct2026.03 | 98 | |
| TIPDefense=No Defense, Victim Model=Llama3.1-8B-Instruct2026.03 | 96 | |
| TIPDefense=Instruction Prevention, Victim Model=Llama3.3-70B-Instruct2026.03 | 92 | |
| TIPDefense=Finetuned Detector, Victim Model=Llama3.3-70B-Instruct2026.03 | 90 | |
| TIPDefense=Perplexity Filtering, Victim Model=Llama3.1-8B-Instruct2026.03 | 90 | |
| TIPDefense=Finetuned Detector, Victim Model=Qwen2.5-72B-Instruct2026.03 | 85 | |
| TIPDefense=Sandwich Prevention, Victim Model=Qwen2.5-72B-Instruct2026.03 | 74 | |
| FixedDefense=Perplexity Filtering, Victim Model=Llama3.3-70B-Instruct2026.03 | 70 | |
| TIPDefense=Sandwich Prevention, Victim Model=Llama3.1-8B-Instruct2026.03 | 69.4 | |
| FixedDefense=No Defense, Victim Model=Llama3.3-70B-Instruct2026.03 | 52 | |
| TIPDefense=Instruction Prevention, Victim Model=Qwen2.5-72B-Instruct2026.03 | 42 | |
| TAPDefense=Finetuned Detector, Victim Model=Llama3.1-8B-Instruct2026.03 | 40 | |
| FixedDefense=Perplexity Filtering, Victim Model=Llama3.1-8B-Instruct2026.03 | 25 | |
| TIPDefense=Sandwich Prevention, Victim Model=Llama3.3-70B-Instruct2026.03 | 23 | |
| TIPDefense=Instruction Prevention, Victim Model=Llama3.1-8B-Instruct2026.03 | 15.5 | |
| TAPDefense=Sandwich Prevention, Victim Model=Llama3.1-8B-Instruct2026.03 | 14 | |
| FixedDefense=No Defense, Victim Model=Llama3.1-8B-Instruct2026.03 | 8.2 | |
| FixedDefense=No Defense, Victim Model=Qwen2.5-72B-Instruct2026.03 | 8.1 | |
| FixedDefense=Finetuned Detector, Victim Model=Llama3.1-8B-Instruct2026.03 | 5 | |
| FixedDefense=Perplexity Filtering, Victim Model=Qwen2.5-72B-Instruct2026.03 | 5 | |
| FixedDefense=Finetuned Detector, Victim Model=Llama3.3-70B-Instruct2026.03 | 3 | |
| FixedDefense=Sandwich Prevention, Victim Model=Llama3.1-8B-Instruct2026.03 | 2.7 | |
| TAPDefense=No Defense, Victim Model=Llama3.1-8B-Instruct2026.03 | 2 | |
| TAPDefense=Perplexity Filtering, Victim Model=Llama3.1-8B-Instruct2026.03 | 2 | |
| FixedDefense=Instruction Prevention, Victim Model=Llama3.3-70B-Instruct2026.03 | 1 | |
| FixedDefense=Sandwich Prevention, Victim Model=Qwen2.5-72B-Instruct2026.03 | 1 | |
| TAPDefense=No Defense, Victim Model=Qwen2.5-72B-Instruct2026.03 | 0 | |
| TAPDefense=No Defense, Victim Model=Llama3.3-70B-Instruct2026.03 | 0 | |
| FixedDefense=Instruction Prevention, Victim Model=Llama3.1-8B-Instruct2026.03 | 0 | |
| TAPDefense=Instruction Prevention, Victim Model=Llama3.1-8B-Instruct2026.03 | 0 | |
| FixedDefense=Instruction Prevention, Victim Model=Qwen2.5-72B-Instruct2026.03 | 0 | |
| TAPDefense=Instruction Prevention, Victim Model=Qwen2.5-72B-Instruct2026.03 | 0 | |
| TAPDefense=Instruction Prevention, Victim Model=Llama3.3-70B-Instruct2026.03 | 0 | |
| TAPDefense=Sandwich Prevention, Victim Model=Qwen2.5-72B-Instruct2026.03 | 0 | |
| FixedDefense=Sandwich Prevention, Victim Model=Llama3.3-70B-Instruct2026.03 | 0 | |
| TAPDefense=Sandwich Prevention, Victim Model=Llama3.3-70B-Instruct2026.03 | 0 | |
| FixedDefense=Finetuned Detector, Victim Model=Qwen2.5-72B-Instruct2026.03 | 0 | |
| TAPDefense=Finetuned Detector, Victim Model=Qwen2.5-72B-Instruct2026.03 | 0 | |
| TAPDefense=Finetuned Detector, Victim Model=Llama3.3-70B-Instruct2026.03 | 0 | |
| TAPDefense=Perplexity Filtering, Victim Model=Qwen2.5-72B-Instruct2026.03 | 0 | |
| TAPDefense=Perplexity Filtering, Victim Model=Llama3.3-70B-Instruct2026.03 | 0 |