Prompt Injection Prevention on Alpaca-Farm
0ASRRobustness via Referencing
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Robustness via ReferencingBase Model=Llama3-8B-Instruct, Attack Type=Ignore2025.04 | 0 | — | — | — | — | — | — | — | |
| Robustness via ReferencingBase Model=Llama3-8B-Instruct, Attack Type=Fakecom2025.04 | 0 | — | — | — | — | — | — | — | |
| Robustness via ReferencingBase Model=Llama3.1-8B-Instruct, Attack Type=Ignore2025.04 | 0 | — | — | — | — | — | — | — | |
| Robustness via ReferencingBase Model=Llama3.1-8B-Instruct, Attack Type=Combined2025.04 | 0.48 | — | — | — | — | — | — | — | |
| StruQBase Model=Llama3-8B-Instruct, Attack Type=Ignore2025.04 | 0.96 | — | — | — | — | — | — | — | |
| StruQBase Model=Llama3.1-8B-Instruct, Attack Type=Ignore2025.04 | 0.96 | — | — | — | — | — | — | — | |
| StruQBase Model=Llama3.1-8B-Instruct, Attack Type=Escape2025.04 | 0.96 | — | — | — | — | — | — | — | |
| Robustness via ReferencingBase Model=Llama3-8B-Instruct, Attack Type=Escape2025.04 | 0.96 | — | — | — | — | — | — | — | |
| Robustness via ReferencingBase Model=Llama3-8B-Instruct, Attack Type=Combined2025.04 | 0.96 | — | — | — | — | — | — | — | |
| Robustness via ReferencingBase Model=Llama3.1-8B-Instruct, Attack Type=Fakecom2025.04 | 0.96 | — | — | — | — | — | — | — | |
| StruQBase Model=Qwen2-7B-Instruct, Attack Type=Escape2025.04 | 1.92 | — | — | — | — | — | — | — | |
| Robustness via ReferencingBase Model=Qwen2-7B-Instruct, Attack Type=Fakecom2025.04 | 1.92 | — | — | — | — | — | — | — | |
| Robustness via ReferencingBase Model=Qwen2-7B-Instruct, Attack Type=Combined2025.04 | 1.92 | — | — | — | — | — | — | — | |
| Robustness via ReferencingBase Model=Llama3.1-8B-Instruct, Attack Type=Escape2025.04 | 1.92 | — | — | — | — | — | — | — | |
| StruQBase Model=Llama3-8B-Instruct, Attack Type=Escape2025.04 | 2.4 | — | — | — | — | — | — | — | |
| StruQBase Model=Llama3-8B-Instruct, Attack Type=Combined2025.04 | 2.4 | — | — | — | — | — | — | — | |
| Robustness via ReferencingBase Model=Qwen2-7B-Instruct, Attack Type=Ignore2025.04 | 2.4 | — | — | — | — | — | — | — | |
| Robustness via ReferencingBase Model=Qwen2-7B-Instruct, Attack Type=Escape2025.04 | 2.4 | — | — | — | — | — | — | — | |
| Robustness via ReferencingBase Model=Llama3.1-8B-Instruct, Attack Type=Naive2025.04 | 2.4 | — | — | — | — | — | — | — | |
| StruQBase Model=Llama3-8B-Instruct, Attack Type=Fakecom2025.04 | 2.88 | — | — | — | — | — | — | — | |
| Robustness via ReferencingBase Model=Llama3-8B-Instruct, Attack Type=Naive2025.04 | 2.88 | — | — | — | — | — | — | — | |
| Robustness via ReferencingBase Model=Qwen2-7B-Instruct, Attack Type=Naive2025.04 | 2.88 | — | — | — | — | — | — | — | |
| StruQBase Model=Llama3.1-8B-Instruct, Attack Type=Naive2025.04 | 4.81 | — | — | — | — | — | — | — | |
| StruQBase Model=Llama3-8B-Instruct, Attack Type=Naive2025.04 | 5.29 | — | — | — | — | — | — | — | |
| StruQBase Model=Qwen2-7B-Instruct, Attack Type=Ignore2025.04 | 9.62 | — | — | — | — | — | — | — | |
| StruQBase Model=Qwen2-7B-Instruct, Attack Type=Naive2025.04 | 10.1 | — | — | — | — | — | — | — | |
| StruQBase Model=Llama3.1-8B-Instruct, Attack Type=Combined2025.04 | 13.46 | — | — | — | — | — | — | — | |
| StruQBase Model=Qwen2-7B-Instruct, Attack Type=Fakecom2025.04 | 16.35 | — | — | — | — | — | — | — | |
| SandwichBase Model=Llama3-8B-Instruct, Attack Type=Escape2025.04 | 20.67 | — | — | — | — | — | — | — | |
| StruQBase Model=Llama3.1-8B-Instruct, Attack Type=Fakecom2025.04 | 22.12 | — | — | — | — | — | — | — | |
| SpotlightBase Model=Llama3-8B-Instruct, Attack Type=Naive2025.04 | 24.04 | — | — | — | — | — | — | — | |
| SpotlightBase Model=Llama3.1-8B-Instruct, Attack Type=Escape2025.04 | 24.04 | — | — | — | — | — | — | — | |
| ReminderBase Model=Llama3-8B-Instruct, Attack Type=Fakecom2025.04 | 24.52 | — | — | — | — | — | — | — | |
| SandwichBase Model=Llama3-8B-Instruct, Attack Type=Fakecom2025.04 | 25 | — | — | — | — | — | — | — | |
| SandwichBase Model=Llama3-8B-Instruct, Attack Type=Naive2025.04 | 25.48 | — | — | — | — | — | — | — | |
| SpotlightBase Model=Llama3.1-8B-Instruct, Attack Type=Naive2025.04 | 25.96 | — | — | — | — | — | — | — | |
| SandwichBase Model=Qwen2-7B-Instruct, Attack Type=Naive2025.04 | 26.44 | — | — | — | — | — | — | — | |
| SandwichBase Model=Llama3.1-8B-Instruct, Attack Type=Escape2025.04 | 26.44 | — | — | — | — | — | — | — | |
| SpotlightBase Model=Llama3-8B-Instruct, Attack Type=Escape2025.04 | 26.44 | — | — | — | — | — | — | — | |
| SandwichBase Model=Qwen2-7B-Instruct, Attack Type=Fakecom2025.04 | 27.88 | — | — | — | — | — | — | — | |
| SandwichBase Model=Llama3.1-8B-Instruct, Attack Type=Naive2025.04 | 28.37 | — | — | — | — | — | — | — | |
| InstructionalBase Model=Llama3-8B-Instruct, Attack Type=Escape2025.04 | 28.37 | — | — | — | — | — | — | — | |
| SandwichBase Model=Qwen2-7B-Instruct, Attack Type=Escape2025.04 | 29.33 | — | — | — | — | — | — | — | |
| StruQBase Model=Qwen2-7B-Instruct, Attack Type=Combined2025.04 | 30.29 | — | — | — | — | — | — | — | |
| SpotlightBase Model=Llama3.1-8B-Instruct, Attack Type=Ignore2025.04 | 32.69 | — | — | — | — | — | — | — | |
| SandwichBase Model=Llama3.1-8B-Instruct, Attack Type=Fakecom2025.04 | 33.65 | — | — | — | — | — | — | — | |
| ReminderBase Model=Llama3-8B-Instruct, Attack Type=Naive2025.04 | 33.65 | — | — | — | — | — | — | — | |
| InstructionalBase Model=Llama3-8B-Instruct, Attack Type=Naive2025.04 | 34.13 | — | — | — | — | — | — | — | |
| SandwichBase Model=Qwen2-7B-Instruct, Attack Type=Ignore2025.04 | 35.58 | — | — | — | — | — | — | — | |
| SpotlightBase Model=Qwen2-7B-Instruct, Attack Type=Naive2025.04 | 35.58 | — | — | — | — | — | — | — | |
| InstructionalBase Model=Llama3.1-8B-Instruct, Attack Type=Naive2025.04 | 36.06 | — | — | — | — | — | — | — | |
| SpotlightBase Model=Llama3-8B-Instruct, Attack Type=Ignore2025.04 | 36.06 | — | — | — | — | — | — | — | |
| SandwichBase Model=Llama3-8B-Instruct, Attack Type=Ignore2025.04 | 37.02 | — | — | — | — | — | — | — | |
| ReminderBase Model=Llama3.1-8B-Instruct, Attack Type=Escape2025.04 | 37.02 | — | — | — | — | — | — | — | |
| InstructionalBase Model=Llama3-8B-Instruct, Attack Type=Ignore2025.04 | 37.02 | — | — | — | — | — | — | — | |
| SandwichBase Model=Qwen2-7B-Instruct, Attack Type=Combined2025.04 | 37.5 | — | — | — | — | — | — | — | |
| ReminderBase Model=Llama3.1-8B-Instruct, Attack Type=Naive2025.04 | 37.98 | — | — | — | — | — | — | — | |
| SandwichBase Model=Llama3-8B-Instruct, Attack Type=Combined2025.04 | 39.9 | — | — | — | — | — | — | — | |
| ReminderBase Model=Llama3-8B-Instruct, Attack Type=Escape2025.04 | 40.38 | — | — | — | — | — | — | — | |
| ReminderBase Model=Llama3.1-8B-Instruct, Attack Type=Fakecom2025.04 | 40.38 | — | — | — | — | — | — | — | |
| InstructionalBase Model=Llama3-8B-Instruct, Attack Type=Fakecom2025.04 | 40.87 | — | — | — | — | — | — | — | |
| InstructionalBase Model=Llama3.1-8B-Instruct, Attack Type=Escape2025.04 | 40.87 | — | — | — | — | — | — | — | |
| SandwichBase Model=Llama3.1-8B-Instruct, Attack Type=Ignore2025.04 | 42.31 | — | — | — | — | — | — | — | |
| SpotlightBase Model=Qwen2-7B-Instruct, Attack Type=Ignore2025.04 | 43.27 | — | — | — | — | — | — | — | |
| SpotlightBase Model=Qwen2-7B-Instruct, Attack Type=Escape2025.04 | 43.27 | — | — | — | — | — | — | — | |
| InstructionalBase Model=Llama3.1-8B-Instruct, Attack Type=Ignore2025.04 | 44.23 | — | — | — | — | — | — | — | |
| NoneBase Model=Llama3-8B-Instruct, Attack Type=Escape2025.04 | 44.71 | — | — | — | — | — | — | — | |
| InstructionalBase Model=Llama3.1-8B-Instruct, Attack Type=Fakecom2025.04 | 46.63 | — | — | — | — | — | — | — | |
| NoneBase Model=Llama3.1-8B-Instruct, Attack Type=Naive2025.04 | 47.12 | — | — | — | — | — | — | — | |
| NoneBase Model=Llama3.1-8B-Instruct, Attack Type=Escape2025.04 | 47.6 | — | — | — | — | — | — | — | |
| InstructionalBase Model=Qwen2-7B-Instruct, Attack Type=Naive2025.04 | 47.6 | — | — | — | — | — | — | — | |
| NoneBase Model=Llama3-8B-Instruct, Attack Type=Naive2025.04 | 48.08 | — | — | — | — | — | — | — | |
| InstructionalBase Model=Qwen2-7B-Instruct, Attack Type=Escape2025.04 | 48.08 | — | — | — | — | — | — | — | |
| SandwichBase Model=Llama3.1-8B-Instruct, Attack Type=Combined2025.04 | 50 | — | — | — | — | — | — | — | |
| SpotlightBase Model=Llama3.1-8B-Instruct, Attack Type=Fakecom2025.04 | 50 | — | — | — | — | — | — | — | |
| NoneBase Model=Qwen2-7B-Instruct, Attack Type=Naive2025.04 | 50.48 | — | — | — | — | — | — | — | |
| NoneBase Model=Qwen2-7B-Instruct, Attack Type=Escape2025.04 | 52.4 | — | — | — | — | — | — | — | |
| ReminderBase Model=Llama3-8B-Instruct, Attack Type=Combined2025.04 | 53.37 | — | — | — | — | — | — | — | |
| InstructionalBase Model=Llama3-8B-Instruct, Attack Type=Combined2025.04 | 54.81 | — | — | — | — | — | — | — | |
| ReminderBase Model=Llama3-8B-Instruct, Attack Type=Ignore2025.04 | 56.73 | — | — | — | — | — | — | — | |
| ReminderBase Model=Llama3.1-8B-Instruct, Attack Type=Ignore2025.04 | 56.73 | — | — | — | — | — | — | — | |
| SpotlightBase Model=Llama3-8B-Instruct, Attack Type=Combined2025.04 | 56.73 | — | — | — | — | — | — | — | |
| ReminderBase Model=Qwen2-7B-Instruct, Attack Type=Naive2025.04 | 58.17 | — | — | — | — | — | — | — | |
| SpotlightBase Model=Llama3.1-8B-Instruct, Attack Type=Combined2025.04 | 58.65 | — | — | — | — | — | — | — | |
| InstructionalBase Model=Qwen2-7B-Instruct, Attack Type=Ignore2025.04 | 59.13 | — | — | — | — | — | — | — | |
| SpotlightBase Model=Llama3-8B-Instruct, Attack Type=Fakecom2025.04 | 61.06 | — | — | — | — | — | — | — | |
| ReminderBase Model=Qwen2-7B-Instruct, Attack Type=Escape2025.04 | 62.5 | — | — | — | — | — | — | — | |
| InstructionalBase Model=Llama3.1-8B-Instruct, Attack Type=Combined2025.04 | 63.94 | — | — | — | — | — | — | — | |
| NoneBase Model=Qwen2-7B-Instruct, Attack Type=Ignore2025.04 | 64.42 | — | — | — | — | — | — | — | |
| NoneBase Model=Llama3-8B-Instruct, Attack Type=Ignore2025.04 | 65.38 | — | — | — | — | — | — | — | |
| NoneBase Model=Llama3.1-8B-Instruct, Attack Type=Ignore2025.04 | 65.87 | — | — | — | — | — | — | — | |
| NoneBase Model=Llama3-8B-Instruct, Attack Type=Fakecom2025.04 | 68.27 | — | — | — | — | — | — | — | |
| ReminderBase Model=Qwen2-7B-Instruct, Attack Type=Ignore2025.04 | 74.04 | — | — | — | — | — | — | — | |
| ReminderBase Model=Llama3.1-8B-Instruct, Attack Type=Combined2025.04 | 74.04 | — | — | — | — | — | — | — | |
| NoneBase Model=Llama3.1-8B-Instruct, Attack Type=Fakecom2025.04 | 74.52 | — | — | — | — | — | — | — | |
| InstructionalBase Model=Qwen2-7B-Instruct, Attack Type=Fakecom2025.04 | 78.37 | — | — | — | — | — | — | — | |
| NoneBase Model=Llama3-8B-Instruct, Attack Type=Combined2025.04 | 79.33 | — | — | — | — | — | — | — | |
| SpotlightBase Model=Qwen2-7B-Instruct, Attack Type=Combined2025.04 | 80.29 | — | — | — | — | — | — | — | |
| NoneBase Model=Llama3.1-8B-Instruct, Attack Type=Combined2025.04 | 82.21 | — | — | — | — | — | — | — | |
| NoneBase Model=Qwen2-7B-Instruct, Attack Type=Combined2025.04 | 84.13 | — | — | — | — | — | — | — |