Prompt Injection Defense on Prompt Injection Attacks (test)
0.9Naive ASROurs-Ignore
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Ours-IgnoreLLM Model=GPT-4o-Latest2024.11 | 0.9 | 0.9 | 0.4 | 0 | 0 | |
| Ours-IgnoreLLM Model=GPT-3.5-Turbo2024.11 | 2.88 | 3.36 | 1.44 | 0.48 | 4.32 | |
| Ours-FakecomLLM Model=GPT-3.5-Turbo2024.11 | 5.57 | 12.01 | 1.44 | 0 | 14.9 | |
| Ours-FakecomLLM Model=GPT-4o-Latest2024.11 | 7.21 | 2.88 | 5.76 | 0.9 | 7.21 | |
| ReminderLLM Model=GPT-3.5-Turbo2024.11 | 11.05 | 10.57 | 10.96 | 9.61 | 26.92 | |
| SandwichLLM Model=GPT-3.5-Turbo2024.11 | 13.94 | 17.3 | 8.65 | 4.32 | 42.3 | |
| ReminderLLM Model=GPT-4o-Latest2024.11 | 14.9 | 27.88 | 17.3 | 89.42 | 82.69 | |
| SandwichLLM Model=GPT-4o-Latest2024.11 | 15.86 | 29.32 | 10.09 | 5.76 | 37.98 | |
| SpotlightLLM Model=GPT-3.5-Turbo2024.11 | 16.34 | 31.73 | 13.46 | 15.38 | 71.15 | |
| SpotlightLLM Model=GPT-4o-Latest2024.11 | 19.71 | 45.67 | 15.38 | 47.11 | 68.75 | |
| IsolationLLM Model=GPT-3.5-Turbo2024.11 | 22.59 | 39.42 | 24.51 | 43.26 | 77.4 | |
| InstructionalLLM Model=GPT-4o-Latest2024.11 | 24.51 | 18.26 | 28.36 | 62.01 | 42.78 | |
| InstructionalLLM Model=GPT-3.5-Turbo2024.11 | 25 | 34.61 | 26.92 | 44.23 | 71.63 | |
| NoneLLM Model=GPT-3.5-Turbo2024.11 | 32.69 | 50.48 | 32.69 | 88.46 | 87.5 | |
| IsolationLLM Model=GPT-4o-Latest2024.11 | 52.4 | 83.17 | 52.88 | 94.71 | 99.51 | |
| NoneLLM Model=GPT-4o-Latest2024.11 | 65.86 | 92.78 | 63.46 | 100 | 100 |