Jailbreaking Tool-Using LLM Agents on AgentDojo
76.25ASRCFD
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CFDTarget Model=Gemini-2.5-pro2026.06 | 76.25 | — | |
| CFDTarget Model=GPT-4.12026.06 | 66.03 | — | |
| Tree of Attacks (ToA)Target Model=Gemini-2.5-pro2026.06 | 58.36 | — | |
| Crescendo (CR)Target Model=GPT-4.12026.06 | 56.6 | — | |
| Tree of Attacks (ToA)Target Model=Mistral-Small-3.22026.06 | 52.83 | — | |
| Crescendo (CR)Target Model=Gemini-2.5-pro2026.06 | 48.11 | — | |
| Role-Play (RP)Target Model=GPT-4.12026.06 | 47.17 | — | |
| Role-Play (RP)Target Model=Gemini-2.5-pro2026.06 | 42.1 | — | |
| Tree of Attacks (ToA)Target Model=GPT-4.12026.06 | 38.36 | — | |
| Prompt Injection (PI)Target Model=Gemini-2.5-pro2026.06 | 37.52 | — | |
| CFDTarget Model=Mistral-Small-3.22026.06 | 37.11 | — | |
| CFDTarget Model=Gemini-2.0-flash2026.06 | 36.85 | — | |
| Crescendo (CR)Target Model=Mistral-Small-3.22026.06 | 36.48 | — | |
| Role-Play (RP)Target Model=Gemini-2.0-flash2026.06 | 35.85 | — | |
| CFDTarget Model=Claude-3.5-Sonnet2026.06 | 31.44 | — | |
| Direct AskingTarget Model=Gemini-2.5-pro2026.06 | 30.19 | — | |
| Tree of Attacks (ToA)Target Model=Gemini-2.0-flash2026.06 | 25.79 | — | |
| Role-Play (RP)Target Model=Mistral-Small-3.22026.06 | 22.01 | — | |
| Prompt Injection (PI)Target Model=GPT-4.12026.06 | 21.38 | — | |
| CFDTarget Model=GPT-5-nano2026.06 | 20.12 | — | |
| Role-Play (RP)Target Model=Claude-3.5-Sonnet2026.06 | 19.5 | — | |
| Crescendo (CR)Target Model=Gemini-2.0-flash2026.06 | 10.06 | — | |
| Direct AskingTarget Model=Mistral-Small-3.22026.06 | 9.43 | — | |
| Crescendo (CR)Target Model=Claude-3.5-Sonnet2026.06 | 7.55 | — | |
| Tree of Attacks (ToA)Target Model=GPT-5-nano2026.06 | 7.55 | — | |
| Direct AskingTarget Model=GPT-5-nano2026.06 | 6.6 | — | |
| Prompt Injection (PI)Target Model=Mistral-Small-3.22026.06 | 4.4 | — | |
| Role-Play (RP)Target Model=GPT-5-nano2026.06 | 3.14 | — | |
| Tree of Attacks (ToA)Target Model=Claude-3.5-Sonnet2026.06 | 3.14 | — | |
| Prompt Injection (PI)Target Model=GPT-5-nano2026.06 | 2.52 | — | |
| Crescendo (CR)Target Model=GPT-5-nano2026.06 | 1.26 | — | |
| Direct AskingTarget Model=GPT-4.12026.06 | 0 | — | |
| Direct AskingTarget Model=Gemini-2.0-flash2026.06 | 0 | — | |
| Direct AskingTarget Model=Claude-3.5-Sonnet2026.06 | 0 | — | |
| Prompt Injection (PI)Target Model=Gemini-2.0-flash2026.06 | 0 | — | |
| Prompt Injection (PI)Target Model=Claude-3.5-Sonnet2026.06 | 0 | — | |
| CFDTarget Model=Average across models2026.06 | — | 1.17 | |
| Crescendo (CR)Target Model=Average across models2026.06 | — | 3.5 | |
| Direct AskingTarget Model=Average across models2026.06 | — | 5.17 | |
| Prompt Injection (PI)Target Model=Average across models2026.06 | — | 5.33 | |
| Role-Play (RP)Target Model=Average across models2026.06 | — | 3.17 | |
| Tree of Attacks (ToA)Target Model=Average across models2026.06 | — | 2.67 |