System Prompt Exfiltration on Cline Agent Environment
100Pseudo-RecallToolLeak
Evaluation Results
| Method | Links | |
|---|---|---|
| ToolLeakBackend LLM=sonnet-42025.09 | 100 | |
| ToolLeakBackend LLM=sonnet-4.52025.09 | 100 | |
| ToolLeakBackend LLM=gemini-2.5-pro2025.09 | 100 | |
| ToolLeakBackend LLM=gpt-52025.09 | 100 | |
| ToolLeakBackend LLM=grok-42025.09 | 81 | |
| NaiveBackend LLM=grok-42025.09 | 62 | |
| Completion-RememberBackend LLM=gemini-2.5-pro2025.09 | 24 | |
| Remember-the-StartBackend LLM=grok-42025.09 | 3 | |
| NaiveBackend LLM=sonnet-42025.09 | 0 | |
| NaiveBackend LLM=sonnet-4.52025.09 | 0 | |
| NaiveBackend LLM=gemini-2.5-pro2025.09 | 0 | |
| NaiveBackend LLM=gpt-52025.09 | 0 | |
| IgnoreBackend LLM=grok-42025.09 | 0 | |
| IgnoreBackend LLM=sonnet-42025.09 | 0 | |
| IgnoreBackend LLM=sonnet-4.52025.09 | 0 | |
| IgnoreBackend LLM=gemini-2.5-pro2025.09 | 0 | |
| IgnoreBackend LLM=gpt-52025.09 | 0 | |
| CompletionBackend LLM=grok-42025.09 | 0 | |
| CompletionBackend LLM=sonnet-42025.09 | 0 | |
| CompletionBackend LLM=sonnet-4.52025.09 | 0 | |
| CompletionBackend LLM=gemini-2.5-pro2025.09 | 0 | |
| CompletionBackend LLM=gpt-52025.09 | 0 | |
| Ignore-CompletionBackend LLM=grok-42025.09 | 0 | |
| Ignore-CompletionBackend LLM=sonnet-42025.09 | 0 | |
| Ignore-CompletionBackend LLM=sonnet-4.52025.09 | 0 | |
| Ignore-CompletionBackend LLM=gemini-2.5-pro2025.09 | 0 | |
| Ignore-CompletionBackend LLM=gpt-52025.09 | 0 | |
| Remember-the-StartBackend LLM=sonnet-42025.09 | 0 | |
| Remember-the-StartBackend LLM=sonnet-4.52025.09 | 0 | |
| Remember-the-StartBackend LLM=gemini-2.5-pro2025.09 | 0 | |
| Remember-the-StartBackend LLM=gpt-52025.09 | 0 | |
| Ignore-RememberBackend LLM=grok-42025.09 | 0 | |
| Ignore-RememberBackend LLM=sonnet-42025.09 | 0 | |
| Ignore-RememberBackend LLM=sonnet-4.52025.09 | 0 | |
| Ignore-RememberBackend LLM=gemini-2.5-pro2025.09 | 0 | |
| Ignore-RememberBackend LLM=gpt-52025.09 | 0 | |
| Completion-RememberBackend LLM=grok-42025.09 | 0 | |
| Completion-RememberBackend LLM=sonnet-42025.09 | 0 | |
| Completion-RememberBackend LLM=sonnet-4.52025.09 | 0 | |
| Completion-RememberBackend LLM=gpt-52025.09 | 0 | |
| Ignore-Completion-RememberBackend LLM=grok-42025.09 | 0 | |
| Ignore-Completion-RememberBackend LLM=sonnet-42025.09 | 0 | |
| Ignore-Completion-RememberBackend LLM=sonnet-4.52025.09 | 0 | |
| Ignore-Completion-RememberBackend LLM=gemini-2.5-pro2025.09 | 0 | |
| Ignore-Completion-RememberBackend LLM=gpt-52025.09 | 0 | |
| PLeakBackend LLM=grok-42025.09 | 0 | |
| PLeakBackend LLM=sonnet-42025.09 | 0 | |
| PLeakBackend LLM=sonnet-4.52025.09 | 0 | |
| PLeakBackend LLM=gemini-2.5-pro2025.09 | 0 | |
| PLeakBackend LLM=gpt-52025.09 | 0 |