Indirect Prompt Injection Red-teaming on Reddit
0ASR (Confidentiality)Claude 3.5 Sonnet
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Claude 3.5 SonnetAgent Category=Adapted LLM-based Agents2025.05 | 0 | 48.21 | 35 | 50 | 50 | 54.76 | |
| Claude 3.7 SonnetAgent Category=Adapted LLM-based Agents2025.05 | 0 | 42.86 | 25 | 45 | 48.81 | 40 | |
| GPT-4oAgent Category=Adapted LLM-based Agents2025.05 | 0 | 90.48 | 53.33 | 88.33 | 95.24 | 86.67 | |
| Claude 3.5 Sonnet | CUAAgent Category=Specialized Computer-Use Agents2025.05 | 0 | 45.24 | 10 | 71.67 | 70.24 | 80 | |
| Claude 3.7 Sonnet | CUAAgent Category=Specialized Computer-Use Agents2025.05 | 0 | 52.38 | 35 | 53.33 | 58.93 | 55 | |
| Operator (w/o checks)Agent Category=Specialized Computer-Use Agents2025.05 | 0 | 19.05 | 15 | 21.67 | 20.83 | 23.33 | |
| OperatorAgent Category=Specialized Computer-Use Agents2025.05 | 0 | 8.33 | 3.33 | 8.33 | 11.31 | 6.67 |