Jailbreak Attack on AdvBench
8,712AASRArtPrompt
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| ArtPromptTarget Model=GPT-3.5-turbo-01252025.02 | 8,712 | — | — | 369 | — | — | — | — | — | — | |
| R2JTarget Model=GPT-3.5-turbo-01252025.02 | 8,654 | — | — | 420 | — | — | — | — | — | — | |
| R2JTarget Model=Llama-2-7b-chat2025.02 | 7,538 | — | — | 379 | — | — | — | — | — | — | |
| PAIRTarget Model=GPT-3.5-turbo-01252025.02 | 6,250 | — | — | 349 | — | — | — | — | — | — | |
| AutoDANTarget Model=Llama-2-7b-chat2025.02 | 5,269 | — | — | 306 | — | — | — | — | — | — | |
| TAPTarget Model=GPT-3.5-turbo-01252025.02 | 5,231 | — | — | 296 | — | — | — | — | — | — | |
| ArtPromptTarget Model=Llama-2-7b-chat2025.02 | 4,346 | — | — | 182 | — | — | — | — | — | — | |
| RedEvalTarget Model=GPT-3.5-turbo-01252025.02 | 3,692 | — | — | 317 | — | — | — | — | — | — | |
| AmpleGCG-plusTarget Model=Llama-2-7b-chat2025.02 | 3,192 | — | — | 372 | — | — | — | — | — | — | |
| TAPTarget Model=Llama-2-7b-chat2025.02 | 2,694 | — | — | 174 | — | — | — | — | — | — | |
| PAIRTarget Model=Llama-2-7b-chat2025.02 | 2,538 | — | — | 180 | — | — | — | — | — | — | |
| AmpleGCG-plusTarget Model=GPT-3.5-turbo-01252025.02 | 692 | — | — | 135 | — | — | — | — | — | — | |
| CoT-2Target Model=GPT-3.5-turbo-01252025.02 | 231 | — | — | 110 | — | — | — | — | — | — | |
| CoTTarget Model=GPT-3.5-turbo-01252025.02 | 115 | — | — | 103 | — | — | — | — | — | — | |
| Cres.Target Model=GPT-4o2026.01 | 100 | — | — | — | — | — | — | — | — | — | |
| LATSTarget Model=GPT-4o, Branching factor (k)=3, Tree depth (d)=3, Retries (Rretry)=22026.01 | 100 | — | — | — | — | — | — | — | — | — | |
| LATSTarget Model=Vicuna-13B, Branching factor (k)=3, Tree depth (d)=3, Retries (Rretry)=22026.01 | 100 | — | — | — | — | — | — | — | — | — | |
| LATSTarget Model=Mistral-7B-Inst, Branching factor (k)=3, Tree depth (d)=3, Retries (Rretry)=22026.01 | 100 | — | — | — | — | — | — | — | — | — | |
| F-SOURTarget Model=DeepSeek-V2-Lite2026.02 | 100 | — | — | — | — | — | — | — | — | — | |
| F-SOURTarget Model=OLMoE-1B-7B2026.02 | 100 | — | — | — | — | — | — | — | — | — | |
| DictAttackTarget Model=gpt-4o, Defense Status=Undefended2025.03 | 99.8 | — | — | — | — | — | — | — | — | — | |
| Cres.Target Model=Mistral-7B-Inst2026.01 | 99.2 | — | — | — | — | — | — | — | — | — | |
| EquaCodeEvaluator=GPT-ASR2025.12 | 98.99 | — | — | — | — | 100 | 100 | 95.96 | 100 | — | |
| MaliciousTarget LLM=Qwen-2.5-7b2026.01 | 98.97 | — | — | — | 4.94 | — | — | — | — | — | |
| MAPATarget Model=Llava V1.6-Mistral-7B2026.02 | 98.96 | — | — | — | — | — | — | — | — | — | |
| DeepInception (Text Baseline)Target Model=Qwen2.5-Omni, Modality=Text2026.01 | 98.85 | — | — | — | — | — | — | — | — | — | |
| MaliciousTarget LLM=GPT-3.52026.01 | 98.71 | — | — | — | 4.88 | — | — | — | — | — | |
| LATSTarget Model=GPT-5.1, Branching factor (k)=3, Tree depth (d)=3, Retries (Rretry)=22026.01 | 98.6 | — | — | — | — | — | — | — | — | — | |
| Cres.Target Model=Mistral-7B2026.01 | 98.4 | — | — | — | — | — | — | — | — | — | |
| PAIRTarget Model=Vicuna-13B2026.01 | 98.3 | — | — | — | — | — | — | — | — | — | |
| Audio Narrative AttacksTarget Model=Gemini 2.0 Flash2026.01 | 98.26 | — | — | — | — | — | — | — | — | — | |
| MaliciousTarget LLM=Qwen-2.5-3b2026.01 | 97.94 | — | — | — | 4.9 | — | — | — | — | — | |
| MAPATarget Model=Qwen2.5-VL-7B-Instruct2026.02 | 97.92 | — | — | — | — | — | — | — | — | — | |
| LATSTarget Model=Mistral-7B, Branching factor (k)=3, Tree depth (d)=3, Retries (Rretry)=22026.01 | 97.6 | — | — | — | — | — | — | — | — | — | |
| LATSTarget Model=Llama-3.1-8B-Inst, Branching factor (k)=3, Tree depth (d)=3, Retries (Rretry)=22026.01 | 97.5 | — | — | — | — | — | — | — | — | — | |
| MaliciousTarget LLM=GPT-4omini2026.01 | 97.16 | — | — | — | 4.92 | — | — | — | — | — | |
| MaliciousTarget LLM=Llama-2-7b2026.01 | 96.91 | — | — | — | 4.9 | — | — | — | — | — | |
| Cres.Target Model=Vicuna-13B2026.01 | 96.8 | — | — | — | — | — | — | — | — | — | |
| CoATarget Model=GPT-4o2026.01 | 96.5 | — | — | — | — | — | — | — | — | — | |
| ActorTarget Model=Vicuna-13B2026.01 | 96.4 | — | — | — | — | — | — | — | — | — | |
| MaliciousTarget LLM=Llama-3-70b2026.01 | 96.39 | — | — | — | 4.93 | — | — | — | — | — | |
| LATSTarget Model=Claude-Sonnet-4.5, Branching factor (k)=3, Tree depth (d)=3, Retries (Rretry)=22026.01 | 96.2 | — | — | — | — | — | — | — | — | — | |
| MAPATarget Model=Average2026.02 | 96.18 | — | — | — | — | — | — | — | — | — | |
| SATA-ELPVictim Model=gpt-3.5-turbo, Selection Strategy=ensemble2024.12 | 96 | — | — | — | 4.96 | — | — | — | — | — | |
| SATA-MLMVictim Model=gpt-3.5-turbo, Selection Strategy=ensemble2024.12 | 96 | — | — | — | 4.94 | — | — | — | — | — | |
| F-SOURTarget Model=Mixtral-8x7B2026.02 | 96 | — | — | — | — | — | — | — | — | — | |
| AutoDAN-TurboTarget Model=gpt-4o, Defense Status=Undefended2025.03 | 96 | — | — | — | — | — | — | — | — | — | |
| TrojanPraiseTarget LLM=Llama-3-8b2026.01 | 95.88 | — | — | — | 4.86 | — | — | — | — | — | |
| ActorTarget Model=GPT-4o2026.01 | 95.8 | — | — | — | — | — | — | — | — | — | |
| MaliciousTarget LLM=Llama-3-8b2026.01 | 95.62 | — | — | — | 4.86 | — | — | — | — | — | |
| Audio Narrative AttacksTarget Model=GPT-4o Realtime2026.01 | 95 | — | — | — | — | — | — | — | — | — | |
| TrojanPraiseTarget LLM=Llama-2-7b2026.01 | 94.85 | — | — | — | 4.79 | — | — | — | — | — | |
| TAPTarget Model=Vicuna-13B2026.01 | 94.6 | — | — | — | — | — | — | — | — | — | |
| TAPTarget Model=GPT-4o2026.01 | 94.2 | — | — | — | — | — | — | — | — | — | |
| SATA-MLMVictim Model=gpt-4o-mini, Selection Strategy=ensemble2024.12 | 94 | — | — | — | 4.88 | — | — | — | — | — | |
| F-SOURTarget Model=Qwen1.5-MoE-A2.7B2026.02 | 94 | — | — | — | — | — | — | — | — | — | |
| TrojanPraiseTarget LLM=GPT-3.52026.01 | 93.04 | — | — | — | 4.43 | — | — | — | — | — | |
| RICEModel=Janus2026.03 | 92.69 | — | — | — | — | — | — | — | — | — | |
| CoATarget Model=Llava V1.6-Mistral-7B2026.02 | 91.67 | — | — | — | — | — | — | — | — | — | |
| MAPATarget Model=Llama-3.2-Vision-11B-Instruct2026.02 | 91.66 | — | — | — | — | — | — | — | — | — | |
| TrojanPraiseTarget LLM=Qwen-2.5-3b2026.01 | 90.46 | — | — | — | 4.62 | — | — | — | — | — | |
| TrojanPraiseTarget LLM=GPT-4omini2026.01 | 90.21 | — | — | — | 4.54 | — | — | — | — | — | |
| SATA-MLMVictim Model=gpt-3.5-turbo, Selection Strategy=top12024.12 | 90 | — | — | — | 4.74 | — | — | — | — | — | |
| Dataset-Level Top-5 RoSaisTarget Model=DeepSeek-V2-Lite2026.02 | 90 | — | — | — | — | — | — | — | — | — | |
| RICEModel=Bagel2026.03 | 89.62 | — | — | — | — | — | — | — | — | — | |
| AdvWaveTarget Model=Gemini 2.0 Flash2026.01 | 89.61 | — | — | — | — | — | — | — | — | — | |
| AdvWaveTarget Model=GPT-4o Realtime2026.01 | 89.03 | — | — | — | — | — | — | — | — | — | |
| Audio Narrative AttacksTarget Model=Gemini 2.0 Flash, Modality=Audio2026.01 | 88.65 | — | — | — | — | — | — | — | — | — | |
| Cres.Target Model=GPT-5.12026.01 | 88.1 | — | — | — | — | — | — | — | — | — | |
| AdvPrompterVictim Model=gpt-3.5-turbo2024.12 | 88 | — | — | — | 4.74 | — | — | — | — | — | |
| Safe2HarmTarget Model=Llama-3-8B-Instruct2025.12 | 87 | — | — | — | — | — | — | — | — | — | |
| ADANTarget Model=Mistral-7B-Inst2026.01 | 86.7 | — | — | — | — | — | — | — | — | — | |
| R2JVictim Model=GPT-3.5-turbo-0125, Avg. Training FLOPS=35604 GFLOPS2025.02 | 86.54 | — | — | 4.2 | — | — | — | — | — | — | |
| DeepInception (Text Baseline)Target Model=Gemini 2.0 Flash, Modality=Text2026.01 | 86.15 | — | — | — | — | — | — | — | — | — | |
| SATA-ELPVictim Model=Claude-v2, Selection Strategy=ensemble2024.12 | 86 | — | — | — | 4.54 | — | — | — | — | — | |
| ActorTarget Model=Llama-3.1-8B-Inst2026.01 | 85.2 | — | — | — | — | — | — | — | — | — | |
| SATA-MLMVictim Model=Average, Selection Strategy=ensemble2024.12 | 85 | — | — | — | 4.57 | — | — | — | — | — | |
| Audio Narrative AttacksTarget Model=Qwen2.5-Omni, Modality=Audio2026.01 | 84.64 | — | — | — | — | — | — | — | — | — | |
| Audio Narrative AttacksTarget Model=Qwen2.5-Omni2026.01 | 83.84 | — | — | — | — | — | — | — | — | — | |
| ActorTarget Model=GPT-5.12026.01 | 82.7 | — | — | — | — | — | — | — | — | — | |
| SATA-MLMVictim Model=Llama-3-70B, Selection Strategy=ensemble2024.12 | 82 | — | — | — | 4.6 | — | — | — | — | — | |
| SATA-MLMVictim Model=gpt-4o, Selection Strategy=ensemble2024.12 | 82 | — | — | — | 4.36 | — | — | — | — | — | |
| TAPTarget Model=Qwen1.5-MoE-A2.7B2026.02 | 82 | — | — | — | — | — | — | — | — | — | |
| CoATarget Model=Average2026.02 | 81.8 | — | — | — | — | — | — | — | — | — | |
| ActorTarget Model=Mistral-7B-Inst2026.01 | 81.5 | — | — | — | — | — | — | — | — | — | |
| CoATarget Model=GPT-5.12026.01 | 81.3 | — | — | — | — | — | — | — | — | — | |
| ActorTarget Model=Mistral-7B2026.01 | 80.7 | — | — | — | — | — | — | — | — | — | |
| Sample-Level Top-5 RoSaisTarget Model=Mixtral-8x7B2026.02 | 80 | — | — | — | — | — | — | — | — | — | |
| Dataset-Level Top-5 RoSaisTarget Model=Mixtral-8x7B2026.02 | 80 | — | — | — | — | — | — | — | — | — | |
| TAPTarget Model=GPT-5.12026.01 | 79.4 | — | — | — | — | — | — | — | — | — | |
| ADANTarget Model=Mistral-7B2026.01 | 79.3 | — | — | — | — | — | — | — | — | — | |
| ADANTarget Model=Vicuna-13B2026.01 | 78.5 | — | — | — | — | — | — | — | — | — | |
| PAIRTarget Model=Mistral-7B-Inst2026.01 | 78.4 | — | — | — | — | — | — | — | — | — | |
| TrojanPraiseTarget LLM=Llama-3-70b2026.01 | 78.09 | — | — | — | 4.19 | — | — | — | — | — | |
| ArtPromptVictim Model=gpt-3.5-turbo, Selection Strategy=ensemble2024.12 | 78 | — | — | — | 4.56 | — | — | — | — | — | |
| ActorTarget Model=Claude-Sonnet-4.52026.01 | 77.3 | — | — | — | — | — | — | — | — | — | |
| CoATarget Model=Llama-3.2-Vision-11B-Instruct2026.02 | 77.08 | — | — | — | — | — | — | — | — | — | |
| FigStep-ProModel=Janus2026.03 | 76.92 | — | — | — | — | — | — | — | — | — | |
| CoATarget Model=Qwen2.5-VL-7B-Instruct2026.02 | 76.66 | — | — | — | — | — | — | — | — | — | |
| DictAttackTarget Model=gpt-4o, Defense Status=Guarded, Guard Type=Dual-Plane Guard (SelfDefend-based; gpt-4o auditor)2025.03 | 75.8 | — | — | — | — | — | — | — | — | — |