Prompt Injection on SQuAD Inj
1.11ASR (Naive)Robustness via Referencing
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Robustness via ReferencingBackbone=Llama3.1-405B-Instruct2025.04 | 1.11 | 0.22 | 0.56 | 0.78 | 0.11 | |
| Robustness via ReferencingBackbone=Llama3-70B-Instruct2025.04 | 2.22 | 1.44 | 1.22 | 0.44 | 1.22 | |
| Robustness via ReferencingBackbone=Qwen2-72B-Instruct2025.04 | 2.78 | 0.22 | 2 | 0.78 | 0 | |
| SandwichBackbone=Llama3.1-405B-Instruct2025.04 | 8.11 | 24.44 | 8.22 | 9.44 | 33.22 | |
| SandwichBackbone=Qwen2-72B-Instruct2025.04 | 9.78 | 15.78 | 7.78 | 3.78 | 13.11 | |
| SandwichBackbone=Llama3-70B-Instruct2025.04 | 10.11 | 32.22 | 8 | 48.33 | 46.33 | |
| SpotlightBackbone=Llama3.1-405B-Instruct2025.04 | 15.44 | 57.89 | 14.44 | 77.67 | 85.67 | |
| ReminderBackbone=Llama3.1-405B-Instruct2025.04 | 19.33 | 32.11 | 19.56 | 22.67 | 42.78 | |
| SpotlightBackbone=Qwen2-72B-Instruct2025.04 | 22.56 | 32.78 | 19.33 | 79.22 | 81.33 | |
| NoneBackbone=Llama3.1-405B-Instruct2025.04 | 22.67 | 72.67 | 26.33 | 60 | 80.78 | |
| InstructionalBackbone=Llama3.1-405B-Instruct2025.04 | 23.56 | 37.44 | 23.89 | 34.11 | 42.56 | |
| SpotlightBackbone=Llama3-70B-Instruct2025.04 | 26 | 67.89 | 29.11 | 97.56 | 99.11 | |
| NoneBackbone=Qwen2-72B-Instruct2025.04 | 35.33 | 82.44 | 31.56 | 74.89 | 91.67 | |
| InstructionalBackbone=Llama3-70B-Instruct2025.04 | 42.33 | 46.89 | 42.56 | 91.22 | 75.44 | |
| ReminderBackbone=Qwen2-72B-Instruct2025.04 | 42.67 | 71.89 | 40.11 | 38 | 69.78 | |
| NoneBackbone=Llama3-70B-Instruct2025.04 | 44.78 | 91.67 | 50.33 | 98.22 | 96.67 | |
| ReminderBackbone=Llama3-70B-Instruct2025.04 | 46.78 | 71.56 | 46.44 | 87.78 | 69.44 | |
| InstructionalBackbone=Qwen2-72B-Instruct2025.04 | 47.67 | 70.78 | 42.78 | 50.56 | 79.33 |