Jailbreak Attack on JailbreakBench (ASR)
100ASRDictAttack
Evaluation results
| Method | Links | |
|---|---|---|
| DictAttackModel=gemini-2.5-pro, Grammar Guardrail=disabled2025.03 | 100 | |
| EnumAttackModel=gpt-4o, Grammar Guardrail=None2025.03 | 100 | |
| DictAttackModel=gpt-4o, Grammar Guardrail=None, k=12025.03 | 100 | |
| DictAttackModel=gemini-2.5-pro, Grammar Guardrail=enabled2025.03 | 98.3 | |
| DictAttackModel=gpt-4o, Grammar Guardrail=llama-guard-3-8b, k=12025.03 | 98.3 | |
| DictAttackModel=deepseek-r1, Grammar Guardrail=disabled2025.03 | 97 | |
| EnumAttackModel=gpt-4o-mini, Grammar Guardrail=None2025.03 | 97 | |
| Audio Narrative AttacksTarget Model=GPT-4o Realtime2026.01 | 96.33 | |
| ICD-SEEDBackbone=Qwen-3-4B2026.04 | 95.77 | |
| DictAttackModel=deepseek-r1, Grammar Guardrail=enabled2025.03 | 95.4 | |
| DictAttackModel=gpt-5, Grammar Guardrail=disabled2025.03 | 95 | |
| DictAttackModel=gpt-4o-mini, Grammar Guardrail=None, k=12025.03 | 94 | |
| DictAttackModel=gpt-5, Grammar Guardrail=enabled2025.03 | 93.4 | |
| Audio Narrative AttacksTarget Model=Gemini 2.0 Flash2026.01 | 93.33 | |
| MAPATarget Model=Llava V1.6-Mistral-7B2026.02 | 93.33 | |
| MAPATarget Model=Qwen2.5-VL-7B-Instruct2026.02 | 93.33 | |
| Prompt+RSTarget Model=GPT-3.5, Model Type=Closed-Source2026.01 | 93 | |
| DictAttackModel=gpt-4o-mini, Grammar Guardrail=llama-guard-3-8b, k=12025.03 | 92.4 | |
| EnumAttackModel=gemini-2.0-flash, Grammar Guardrail=None2025.03 | 92 | |
| DictAttackModel=gpt-oss-120b, Grammar Guardrail=disabled2025.03 | 91 | |
| DictAttackModel=gemini-2.0-flash, Grammar Guardrail=None, k=12025.03 | 91 | |
| ICD-SEEDBackbone=Qwen-2.5-3B2026.04 | 90.96 | |
| ICD-PREFILLBackbone=DeepSeek-R1-Distill-Llama-70B2026.04 | 90.58 | |
| Prompt+RSTarget Model=LLaMA2-7B, Model Type=Open-Source2026.01 | 90 | |
| ICD-PREFILLBackbone=Qwen-2.5-3B2026.04 | 89.81 | |
| ICD-SEEDBackbone=Qwen-2.5-7B2026.04 | 89.81 | |
| DictAttackModel=gemini-2.0-flash, Grammar Guardrail=llama-guard-3-8b, k=12025.03 | 89.5 | |
| DictAttackModel=gpt-oss-120b, Grammar Guardrail=enabled2025.03 | 89.4 | |
| ICD-PREFILLBackbone=DeepSeek-R1-Distill-Llama-8B2026.04 | 89.04 | |
| AdvWaveTarget Model=Gemini 2.0 Flash2026.01 | 89 | |
| ICD-SEEDTarget Model=Vicuna-1.5-13B2026.04 | 89 | |
| MAPATarget Model=Average2026.02 | 88.89 | |
| PrefillBackbone=DeepSeek-R1-Distill-Qwen-32B2026.04 | 88.46 | |
| PrefillBackbone=Llama-3.3-70B2026.04 | 87.5 | |
| ICD-SEEDBackbone=DeepSeek-R1-Distill-Llama-8B2026.04 | 87.12 | |
| AdvWaveTarget Model=GPT-4o Realtime2026.01 | 87 | |
| ICD-PREFILLBackbone=DeepSeek-R1-Distill-Qwen-32B2026.04 | 86.92 | |
| ICD-PREFILLBackbone=DeepSeek-R1-Distill-Qwen-14B2026.04 | 86.54 | |
| ICD-PREFILLBackbone=Qwen-3-8B2026.04 | 85.58 | |
| ICD-PREFILLBackbone=DeepSeek-R1-Distill-Qwen-7B2026.04 | 85.38 | |
| ICD-SEEDBackbone=Llama-3.1-8B2026.04 | 85.19 | |
| ICD-PREFILLBackbone=Qwen-2.5-7B2026.04 | 85.19 | |
| ICD-SEEDBackbone=Qwen-3-8B2026.04 | 85.19 | |
| ICD-PREFILLBackbone=Llama-3.1-8B2026.04 | 85 | |
| PrefillBackbone=DeepSeek-R1-Distill-Llama-8B2026.04 | 84.62 | |
| PrefillBackbone=DeepSeek-R1-Distill-Llama-70B2026.04 | 84.23 | |
| ICD-PREFILLTarget Model=Vicuna-1.5-13B2026.04 | 83 | |
| ICD-PREFILLBackbone=Llama-3.3-70B2026.04 | 82.88 | |
| ICD-PREFILLBackbone=Qwen-3-4B2026.04 | 82.69 | |
| RICEModel=Bagel2026.03 | 82 | |
| RICEModel=Janus2026.03 | 82 | |
| PrefillBackbone=Llama-3.2-3B2026.04 | 81.34 | |
| PrefillBackbone=DeepSeek-R1-Distill-Qwen-14B2026.04 | 81.15 | |
| ICD-SEEDBackbone=Qwen-3-14B2026.04 | 80.38 | |
| AdvWaveTarget Model=Qwen2.5-Omni2026.01 | 80 | |
| MAPATarget Model=Llama-3.2-Vision-11B-Instruct2026.02 | 80 | |
| PrefillBackbone=DeepSeek-R1-Distill-Qwen-7B2026.04 | 80 | |
| Safe2HarmTarget Model=Llama-3-8B-Instruct2025.12 | 79.8 | |
| CoATarget Model=Llava V1.6-Mistral-7B2026.02 | 78.33 | |
| PrefillBackbone=Qwen-3-14B2026.04 | 78.27 | |
| Prompt+RSTarget Model=GPT-4, Model Type=Closed-Source2026.01 | 78 | |
| PAIRTarget Model=Llama-3.1-70B2026.04 | 78 | |
| ICD-SEEDBackbone=Qwen-3-32B2026.04 | 77.5 | |
| ICD-SEEDBackbone=DeepSeek-R1-Distill-Qwen-32B2026.04 | 77.31 | |
| PrefillBackbone=Qwen-2.5-7B2026.04 | 77.12 | |
| ICD-PREFILLBackbone=Qwen-2.5-72B2026.04 | 76.35 | |
| PrefillBackbone=Llama-3.1-8B2026.04 | 75.96 | |
| ICD-PREFILLBackbone=Qwen-2.5-32B2026.04 | 75.58 | |
| FigStep-ProModel=Janus2026.03 | 75 | |
| ICD-PREFILLBackbone=Llama-3.2-3B2026.04 | 74.23 | |
| PlainModel=Janus2026.03 | 74 | |
| ICD-PREFILLBackbone=Qwen-2.5-14B2026.04 | 73.27 | |
| Audio Narrative AttacksTarget Model=Qwen2.5-Omni2026.01 | 73 | |
| PrefillBackbone=Qwen-2.5-3B2026.04 | 72.69 | |
| FigStepModel=Janus2026.03 | 72 | |
| ICD-PREFILLBackbone=Gemma-3-27B2026.04 | 71.77 | |
| ICD-AUTOBackbone=Llama-3.1-8B2026.04 | 71.73 | |
| ICD-PREFILLBackbone=Gemma-3-12B2026.04 | 71.35 | |
| PAIRTarget Model=GPT-3.5, Model Type=Closed-Source2026.01 | 71 | |
| FigStep-ProModel=Bagel2026.03 | 71 | |
| CoATarget Model=Qwen2.5-VL-7B-Instruct2026.02 | 70 | |
| ICD-PREFILLBackbone=Qwen-3-14B2026.04 | 68.85 | |
| ICD-AUTOBackbone=Qwen-2.5-3B2026.04 | 68.27 | |
| ICD-AUTOBackbone=Qwen-2.5-7B2026.04 | 68.08 | |
| FigStepModel=Bagel2026.03 | 68 | |
| AMATarget Model=Llama-3.1-70B2026.04 | 68 | |
| ICD-SEEDBackbone=DeepSeek-R1-Distill-Qwen-7B2026.04 | 67.88 | |
| CoATarget Model=Average2026.02 | 67.78 | |
| ICD-SEEDBackbone=Gemma-3-4B2026.04 | 67.12 | |
| FootInTheDoorTarget Model=Llava V1.6-Mistral-7B2026.02 | 66.66 | |
| PrefillBackbone=Gemma-3-12B2026.04 | 66.54 | |
| ICD-SEEDBackbone=Gemma-3-12B2026.04 | 66.16 | |
| ICD-SEEDTarget Model=Llama-3.1-70B2026.04 | 66 | |
| GCGModel=Bagel2026.03 | 65 | |
| PrefillBackbone=Qwen-2.5-32B2026.04 | 64.81 | |
| ICD-SEEDBackbone=Qwen-2.5-14B2026.04 | 64.42 | |
| ICD-SEEDBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.04 | 64.23 | |
| PAIRModel=Bagel2026.03 | 64 | |
| PrefillBackbone=Gemma-3-4B2026.04 | 63.85 | |
| ICD-PREFILLBackbone=Gemma-3-4B2026.04 | 63.46 |