Targeted Prompt Injection on GetWeather
100Attack Success RateTIP
Evaluation results
| Method | Links | |
|---|---|---|
| TIPDefense=No Defense, Victim Model=Qwen2.5-72B-Instruct2026.03 | 100 | |
| TIPDefense=No Defense, Victim Model=Llama3.3-70B-Instruct2026.03 | 100 | |
| TIPDefense=Perplexity Filtering, Victim Model=Qwen2.5-72B-Instruct2026.03 | 100 | |
| TIPDefense=Perplexity Filtering, Victim Model=Llama3.3-70B-Instruct2026.03 | 100 | |
| TIPDefense=No Defense, Victim Model=Llama3.1-8B-Instruct2026.03 | 97.7 | |
| TAPDefense=Instruction Prevention, Victim Model=Qwen2.5-72B-Instruct2026.03 | 94 | |
| TIPDefense=Sandwich Prevention, Victim Model=Qwen2.5-72B-Instruct2026.03 | 93 | |
| TAPDefense=No Defense, Victim Model=Qwen2.5-72B-Instruct2026.03 | 92 | |
| TIPDefense=Instruction Prevention, Victim Model=Qwen2.5-72B-Instruct2026.03 | 92 | |
| TAPDefense=Perplexity Filtering, Victim Model=Qwen2.5-72B-Instruct2026.03 | 92 | |
| TIPDefense=Perplexity Filtering, Victim Model=Llama3.1-8B-Instruct2026.03 | 86 | |
| TIPDefense=Sandwich Prevention, Victim Model=Llama3.1-8B-Instruct2026.03 | 82.9 | |
| TIPDefense=Instruction Prevention, Victim Model=Llama3.1-8B-Instruct2026.03 | 81.5 | |
| TIPDefense=Finetuned Detector, Victim Model=Qwen2.5-72B-Instruct2026.03 | 73 | |
| TIPDefense=Finetuned Detector, Victim Model=Llama3.3-70B-Instruct2026.03 | 73 | |
| TAPDefense=Perplexity Filtering, Victim Model=Llama3.1-8B-Instruct2026.03 | 72 | |
| TIPDefense=Instruction Prevention, Victim Model=Llama3.3-70B-Instruct2026.03 | 67.7 | |
| FixedDefense=Perplexity Filtering, Victim Model=Qwen2.5-72B-Instruct2026.03 | 65 | |
| FixedDefense=No Defense, Victim Model=Qwen2.5-72B-Instruct2026.03 | 64 | |
| TIPDefense=Finetuned Detector, Victim Model=Llama3.1-8B-Instruct2026.03 | 60 | |
| FixedDefense=Instruction Prevention, Victim Model=Qwen2.5-72B-Instruct2026.03 | 52 | |
| TAPDefense=Perplexity Filtering, Victim Model=Llama3.3-70B-Instruct2026.03 | 50 | |
| TAPDefense=Finetuned Detector, Victim Model=Qwen2.5-72B-Instruct2026.03 | 49 | |
| TIPDefense=Sandwich Prevention, Victim Model=Llama3.3-70B-Instruct2026.03 | 47 | |
| TAPDefense=No Defense, Victim Model=Llama3.3-70B-Instruct2026.03 | 39 | |
| FixedDefense=Sandwich Prevention, Victim Model=Qwen2.5-72B-Instruct2026.03 | 37 | |
| FixedDefense=Finetuned Detector, Victim Model=Qwen2.5-72B-Instruct2026.03 | 32 | |
| FixedDefense=Perplexity Filtering, Victim Model=Llama3.3-70B-Instruct2026.03 | 22 | |
| TAPDefense=Finetuned Detector, Victim Model=Llama3.3-70B-Instruct2026.03 | 10 | |
| TAPDefense=Sandwich Prevention, Victim Model=Qwen2.5-72B-Instruct2026.03 | 8 | |
| FixedDefense=Sandwich Prevention, Victim Model=Llama3.1-8B-Instruct2026.03 | 7.3 | |
| TAPDefense=Instruction Prevention, Victim Model=Llama3.1-8B-Instruct2026.03 | 6.8 | |
| FixedDefense=Finetuned Detector, Victim Model=Llama3.1-8B-Instruct2026.03 | 5 | |
| TAPDefense=Finetuned Detector, Victim Model=Llama3.1-8B-Instruct2026.03 | 5 | |
| FixedDefense=Instruction Prevention, Victim Model=Llama3.1-8B-Instruct2026.03 | 4.3 | |
| TAPDefense=Instruction Prevention, Victim Model=Llama3.3-70B-Instruct2026.03 | 3 | |
| FixedDefense=Perplexity Filtering, Victim Model=Llama3.1-8B-Instruct2026.03 | 3 | |
| FixedDefense=No Defense, Victim Model=Llama3.3-70B-Instruct2026.03 | 2 | |
| FixedDefense=No Defense, Victim Model=Llama3.1-8B-Instruct2026.03 | 1.1 | |
| TAPDefense=Sandwich Prevention, Victim Model=Llama3.1-8B-Instruct2026.03 | 1 | |
| TAPDefense=Sandwich Prevention, Victim Model=Llama3.3-70B-Instruct2026.03 | 1 | |
| TAPDefense=No Defense, Victim Model=Llama3.1-8B-Instruct2026.03 | 0 | |
| FixedDefense=Instruction Prevention, Victim Model=Llama3.3-70B-Instruct2026.03 | 0 | |
| FixedDefense=Sandwich Prevention, Victim Model=Llama3.3-70B-Instruct2026.03 | 0 | |
| FixedDefense=Finetuned Detector, Victim Model=Llama3.3-70B-Instruct2026.03 | 0 |