Adversarial Attack Success Rate Assessment on AgentDojo
0ASRChatInject
Evaluation Results
| Method | Links | |
|---|---|---|
| ChatInjectTarget Model=Qwen2-Max, Defense=Injection Detector2026.02 | 0 | |
| ChatInjectTarget Model=DeepSeek-V3, Defense=w/o Defense2026.02 | 0 | |
| ChatInjectTarget Model=DeepSeek-V3, Defense=Injection Detector2026.02 | 0 | |
| GPT-ossPrompt=InjecPrompt, Technique=default2025.09 | 0.3 | |
| GLM-4.5Prompt=InjecPrompt, Technique=default2025.09 | 0.3 | |
| Llama-4Prompt=InjecPrompt, Technique=default2025.09 | 1 | |
| ChatInjectTarget Model=DeepSeek-V3, Defense=Delimiter Spotlighting2026.02 | 1.3 | |
| Llama-4Prompt=Multi-turn, Technique=default2025.09 | 1.8 | |
| PhantomTarget Model=Gemini-1.5-Flash, Defense=Injection Detector2026.02 | 2.6 | |
| PhantomTarget Model=Gemini-1.5-Flash, Defense=Tag Filter2026.02 | 3.25 | |
| Single-TemplateTarget Model=DeepSeek-V3, Defense=Tag Filter2026.02 | 3.25 | |
| ChatInjectTarget Model=DeepSeek-V3, Defense=Tag Filter2026.02 | 3.25 | |
| GPT-ossPrompt=Multi-turn, Technique=default2025.09 | 3.6 | |
| Single-TemplateTarget Model=Qwen2-Max, Defense=Injection Detector2026.02 | 4.55 | |
| Semantic-InjectionTarget Model=Qwen2-Max, Defense=Injection Detector2026.02 | 5.84 | |
| Kimi-K2Prompt=InjecPrompt, Technique=default2025.09 | 5.9 | |
| Grok-2Prompt=InjecPrompt, Technique=default2025.09 | 6.1 | |
| Single-TemplateTarget Model=DeepSeek-V3, Defense=Injection Detector2026.02 | 7.14 | |
| Semantic-InjectionTarget Model=DeepSeek-V3, Defense=Injection Detector2026.02 | 8.44 | |
| PhantomTarget Model=Gemini-1.5-Flash, Defense=w/o Defense2026.02 | 9.74 | |
| PhantomTarget Model=Gemini-1.5-Flash, Defense=Delimiter Spotlighting2026.02 | 10.39 | |
| PhantomTarget Model=Qwen2-Max, Defense=Injection Detector2026.02 | 11.04 | |
| Llama-4Prompt=Multi-turn, Technique=ChatInject2025.09 | 11.1 | |
| Kimi-K2Prompt=Multi-turn, Technique=default2025.09 | 12.3 | |
| PhantomTarget Model=Qwen2-Flash, Defense=Injection Detector2026.02 | 12.34 | |
| PhantomTarget Model=GPT-4o, Defense=Injection Detector2026.02 | 12.34 | |
| PhantomTarget Model=Gemini-1.0-Flash, Defense=Injection Detector2026.02 | 13.64 | |
| ChatInjectTarget Model=Qwen2-Max, Defense=Tag Filter2026.02 | 13.64 | |
| Kimi-K2Prompt=Multi-turn, Technique=ChatInject2025.09 | 13.9 | |
| Single-TemplateTarget Model=Qwen2-Max, Defense=Tag Filter2026.02 | 14.29 | |
| PhantomTarget Model=DeepSeek-V3, Defense=Injection Detector2026.02 | 14.94 | |
| PhantomTarget Model=GPT-4o-Mini, Defense=Injection Detector2026.02 | 15.58 | |
| Llama-4Prompt=InjecPrompt, Technique=ChatInject2025.09 | 17.2 | |
| Qwen-3Prompt=InjecPrompt, Technique=default2025.09 | 17.5 | |
| GLM-4.5Prompt=Multi-turn, Technique=default2025.09 | 17.5 | |
| PhantomTarget Model=GPT-4, Defense=Injection Detector2026.02 | 18.18 | |
| Semantic-InjectionTarget Model=Qwen2-Max, Defense=Delimiter Spotlighting2026.02 | 18.18 | |
| Grok-2Prompt=InjecPrompt, Technique=ChatInject2025.09 | 19.3 | |
| Llama-4Prompt=InjecPrompt, Technique=ChatInject + tool2025.09 | 19.8 | |
| GLM-4.5Prompt=InjecPrompt, Technique=ChatInject2025.09 | 20.3 | |
| Semantic-InjectionTarget Model=Qwen2-Max, Defense=Tag Filter2026.02 | 20.78 | |
| PhantomTarget Model=Gemini-1.0-Flash, Defense=Tag Filter2026.02 | 22.08 | |
| Grok-2Prompt=Multi-turn, Technique=default2025.09 | 23.7 | |
| Semantic-InjectionTarget Model=Qwen2-Max, Defense=w/o Defense2026.02 | 24.68 | |
| Grok-2Prompt=Multi-turn, Technique=ChatInject2025.09 | 24.7 | |
| GLM-4.5Prompt=InjecPrompt, Technique=ChatInject + think2025.09 | 24.8 | |
| ChatInjectTarget Model=Qwen2-Max, Defense=Delimiter Spotlighting2026.02 | 25.32 | |
| Kimi-K2Prompt=InjecPrompt, Technique=ChatInject2025.09 | 29.3 | |
| PhantomTarget Model=GPT-4o, Defense=Tag Filter2026.02 | 31.17 | |
| ChatInjectTarget Model=Qwen2-Max, Defense=w/o Defense2026.02 | 35.71 | |
| GLM-4.5Prompt=InjecPrompt, Technique=ChatInject + tool2025.09 | 36 | |
| PhantomTarget Model=DeepSeek-V3, Defense=Tag Filter2026.02 | 40.26 | |
| PhantomTarget Model=Gemini-1.0-Flash, Defense=Delimiter Spotlighting2026.02 | 40.91 | |
| PhantomTarget Model=Qwen2-Max, Defense=Tag Filter2026.02 | 41.56 | |
| Kimi-K2Prompt=InjecPrompt, Technique=ChatInject + tool2025.09 | 44.2 | |
| Semantic-InjectionTarget Model=DeepSeek-V3, Defense=Tag Filter2026.02 | 47.4 | |
| GPT-ossPrompt=InjecPrompt, Technique=ChatInject + tool2025.09 | 47.4 | |
| GLM-4.5Prompt=Multi-turn, Technique=ChatInject2025.09 | 48.1 | |
| GPT-ossPrompt=InjecPrompt, Technique=ChatInject + think2025.09 | 48.6 | |
| PhantomTarget Model=Qwen2-Flash, Defense=Tag Filter2026.02 | 50 | |
| GPT-ossPrompt=InjecPrompt, Technique=ChatInject2025.09 | 51.4 | |
| Single-TemplateTarget Model=Qwen2-Max, Defense=Delimiter Spotlighting2026.02 | 51.95 | |
| Semantic-InjectionTarget Model=DeepSeek-V3, Defense=w/o Defense2026.02 | 52.6 | |
| PhantomTarget Model=GPT-4o, Defense=w/o Defense2026.02 | 53.25 | |
| PhantomTarget Model=GPT-4o, Defense=Delimiter Spotlighting2026.02 | 53.25 | |
| Single-TemplateTarget Model=Qwen2-Max, Defense=w/o Defense2026.02 | 53.25 | |
| Semantic-InjectionTarget Model=DeepSeek-V3, Defense=Delimiter Spotlighting2026.02 | 53.25 | |
| PhantomTarget Model=Qwen2-Flash, Defense=Delimiter Spotlighting2026.02 | 54.55 | |
| Qwen-3Prompt=InjecPrompt, Technique=ChatInject2025.09 | 54.8 | |
| GPT-ossPrompt=Multi-turn, Technique=ChatInject2025.09 | 55.5 | |
| PhantomTarget Model=Qwen2-Flash, Defense=w/o Defense2026.02 | 55.84 | |
| Qwen-3Prompt=Multi-turn, Technique=default2025.09 | 60.9 | |
| PhantomTarget Model=Qwen2-Max, Defense=Delimiter Spotlighting2026.02 | 63.64 | |
| Qwen-3Prompt=InjecPrompt, Technique=ChatInject + think2025.09 | 66.1 | |
| Single-TemplateTarget Model=DeepSeek-V3, Defense=w/o Defense2026.02 | 66.23 | |
| PhantomTarget Model=Qwen2-Max, Defense=w/o Defense2026.02 | 67.53 | |
| PhantomTarget Model=GPT-4o-Mini, Defense=w/o Defense2026.02 | 67.53 | |
| Single-TemplateTarget Model=DeepSeek-V3, Defense=Delimiter Spotlighting2026.02 | 68.67 | |
| Qwen-3Prompt=InjecPrompt, Technique=ChatInject + tool2025.09 | 69.4 | |
| PhantomTarget Model=GPT-4o-Mini, Defense=Delimiter Spotlighting2026.02 | 69.48 | |
| PhantomTarget Model=GPT-4o-Mini, Defense=Tag Filter2026.02 | 69.48 | |
| PhantomTarget Model=Gemini-1.0-Flash, Defense=w/o Defense2026.02 | 72.73 | |
| PhantomTarget Model=GPT-4, Defense=Tag Filter2026.02 | 74.03 | |
| PhantomTarget Model=GPT-4, Defense=w/o Defense2026.02 | 75.97 | |
| PhantomTarget Model=GPT-4, Defense=Delimiter Spotlighting2026.02 | 76.62 | |
| PhantomTarget Model=DeepSeek-V3, Defense=w/o Defense2026.02 | 78.57 | |
| PhantomTarget Model=DeepSeek-V3, Defense=Delimiter Spotlighting2026.02 | 79.87 | |
| Qwen-3Prompt=Multi-turn, Technique=ChatInject2025.09 | 80.5 |