Indirect Prompt Injection Defense on SQuAD
0.68ASRCachePrune
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| CachePruneModel=Mistral-7B2025.04 | 0.68 | 24.46 | 23.1 | |
| CachePruneModel=Phi-3.5-mini-instruct (3.8B)2025.04 | 0.71 | 26.76 | 25.55 | |
| Encode_Base64Model=Phi-3.5-mini-instruct (3.8B)2025.04 | 0.86 | 7.87 | 4.52 | |
| DatamarkingModel=Phi-3.5-mini-instruct (3.8B)2025.04 | 3.54 | 26.71 | 26.47 | |
| Encode_Base64Model=Mistral-7B2025.04 | 4.78 | 15.32 | 9.56 | |
| DelimitingModel=Mistral-7B2025.04 | 5.28 | 24.38 | 20.07 | |
| DatamarkingModel=Mistral-7B2025.04 | 6.37 | 23.56 | 21.34 | |
| Encode_Base64Model=LLama3-8B2025.04 | 6.56 | 13.34 | 11.56 | |
| CachePruneModel=LLama3-8B2025.04 | 7.44 | 28.68 | 22.84 | |
| CachePruneBase Model=LLama3-8B, Attack Type=Adaptive Attack2025.04 | 7.71 | — | 20.72 | |
| DelimitingModel=Phi-3.5-mini-instruct (3.8B)2025.04 | 7.87 | 26.05 | 25.49 | |
| VanillaModel=Mistral-7B2025.04 | 9.01 | 22.78 | 19.04 | |
| VanillaModel=Phi-3.5-mini-instruct (3.8B)2025.04 | 10.22 | 26.03 | 25.64 | |
| SandwichModel=Mistral-7B2025.04 | 10.36 | 20.25 | 18.33 | |
| DatamarkingModel=LLama3-8B2025.04 | 13.25 | 28.56 | 21.45 | |
| SandwichModel=Phi-3.5-mini-instruct (3.8B)2025.04 | 18.65 | 23.78 | 23.13 | |
| SandwichModel=LLama3-8B2025.04 | 21.43 | 27.69 | 18.98 | |
| DelimitingModel=LLama3-8B2025.04 | 23.6 | 29.34 | 20.56 | |
| VanillaModel=LLama3-8B2025.04 | 27.86 | 28.2 | 19.56 | |
| VanillaBase Model=LLama3-8B, Attack Type=Adaptive Attack2025.04 | 97.89 | — | 1.26 | |
| DaramarkingBase Model=LLama3-8B, Attack Type=Adaptive Attack2025.04 | 99.87 | — | 2.32 | |
| DelimitingBase Model=LLama3-8B, Attack Type=Adaptive Attack2025.04 | 100 | — | 0.57 |