Attack Success Rate (ASR) on AdvBench (Jailbreak Attack)
100ASREnumAttack
Evaluation results
| Method | Links | |
|---|---|---|
| EnumAttackModel=gpt-4o, Grammar Guardrail=None2025.03 | 100 | |
| I-GCGTarget Model=Llama-2-7b, Suffix Length=202025.09 | 100 | |
| I-GCGTarget Model=Llama-2-7b, Suffix Length=302025.09 | 100 | |
| I-GCG + Mask-GCGTarget Model=Llama-2-7b, Suffix Length=202025.09 | 100 | |
| I-GCG + Mask-GCGTarget Model=Llama-2-7b, Suffix Length=302025.09 | 100 | |
| GCGTarget Model=Vicuna-7b, Suffix Length=302025.09 | 100 | |
| I-GCGTarget Model=Vicuna-7b, Suffix Length=202025.09 | 100 | |
| I-GCGTarget Model=Vicuna-7b, Suffix Length=302025.09 | 100 | |
| I-GCG + Mask-GCGTarget Model=Vicuna-7b, Suffix Length=202025.09 | 100 | |
| I-GCG + Mask-GCGTarget Model=Vicuna-7b, Suffix Length=302025.09 | 100 | |
| AmpleGCGTarget Model=Vicuna-7b, Suffix Length=302025.09 | 100 | |
| AmpleGCG + Mask-GCGTarget Model=Vicuna-7b, Suffix Length=302025.09 | 100 | |
| I-GCGTarget Model=Llama-2-13b, Suffix Length=202025.09 | 100 | |
| I-GCGTarget Model=Llama-2-13b, Suffix Length=302025.09 | 100 | |
| I-GCG + Mask-GCGTarget Model=Llama-2-13b, Suffix Length=302025.09 | 100 | |
| AmpleGCGTarget Model=Llama-2-13b, Suffix Length=202025.09 | 100 | |
| AmpleGCGTarget Model=Llama-2-13b, Suffix Length=302025.09 | 100 | |
| AmpleGCG + Mask-GCGTarget Model=Llama-2-13b, Suffix Length=202025.09 | 100 | |
| I-GCGTarget Model=Average, Suffix Length=202025.09 | 100 | |
| I-GCGTarget Model=Average, Suffix Length=302025.09 | 100 | |
| I-GCG + Mask-GCGTarget Model=Average, Suffix Length=302025.09 | 100 | |
| DictAttackModel=gpt-4o, Grammar Guardrail=None, k=12025.03 | 99.8 | |
| ICD-SEEDTarget Model=Vicuna-1.5-13B2026.04 | 99.62 | |
| DictAttackModel=gemini-2.5-pro, Grammar Guardrail=disabled2025.03 | 99.2 | |
| TemplateFuzzModel=Gemma3-4B, Attack Variant=Top-1 Template2026.04 | 99.04 | |
| EnumAttackModel=gpt-4o-mini, Grammar Guardrail=None2025.03 | 99 | |
| AmpleGCGTarget Model=Vicuna-7b, Suffix Length=202025.09 | 99 | |
| I-GCG + Mask-GCGTarget Model=Average, Suffix Length=202025.09 | 99 | |
| AmpleGCGTarget Model=Average, Suffix Length=302025.09 | 99 | |
| AmpleGCG + Mask-GCGTarget Model=Average, Suffix Length=202025.09 | 99 | |
| AmpleGCG + Mask-GCGTarget Model=Average, Suffix Length=302025.09 | 99 | |
| DictAttackModel=deepseek-r1, Grammar Guardrail=disabled2025.03 | 98.9 | |
| TemplateFuzzModel=DeepSeek-7B, Attack Variant=Top-1 Template2026.04 | 98.84 | |
| DictAttackModel=gemini-2.0-flash, Grammar Guardrail=None, k=12025.03 | 98.5 | |
| DictAttackModel=gpt-oss-120b, Grammar Guardrail=disabled2025.03 | 98.3 | |
| DictAttackModel=gpt-4o, Grammar Guardrail=llama-guard-3-8b, k=12025.03 | 98.1 | |
| AmpleGCGTarget Model=Llama-2-7b, Suffix Length=302025.09 | 98 | |
| AmpleGCG + Mask-GCGTarget Model=Llama-2-7b, Suffix Length=202025.09 | 98 | |
| AmpleGCG + Mask-GCGTarget Model=Llama-2-7b, Suffix Length=302025.09 | 98 | |
| GCGTarget Model=Vicuna-7b, Suffix Length=202025.09 | 98 | |
| AmpleGCG + Mask-GCGTarget Model=Vicuna-7b, Suffix Length=202025.09 | 98 | |
| I-GCG + Mask-GCGTarget Model=Llama-2-13b, Suffix Length=202025.09 | 98 | |
| AmpleGCG + Mask-GCGTarget Model=Llama-2-13b, Suffix Length=302025.09 | 98 | |
| DictAttackModel=gemini-2.5-pro, Grammar Guardrail=enabled2025.03 | 97.5 | |
| DictAttackModel=deepseek-r1, Grammar Guardrail=enabled2025.03 | 97.2 | |
| DictAttackModel=gpt-4o-mini, Grammar Guardrail=None, k=12025.03 | 97.1 | |
| AmpleGCGTarget Model=Average, Suffix Length=202025.09 | 97 | |
| DictAttackModel=gemini-2.0-flash, Grammar Guardrail=llama-guard-3-8b, k=12025.03 | 96.8 | |
| DictAttackModel=gpt-oss-120b, Grammar Guardrail=enabled2025.03 | 96.6 | |
| DictAttackModel=gpt-5, Grammar Guardrail=disabled2025.03 | 96 | |
| GCG + Mask-GCGTarget Model=Vicuna-7b, Suffix Length=202025.09 | 96 | |
| GCG + Mask-GCGTarget Model=Vicuna-7b, Suffix Length=302025.09 | 96 | |
| I-GCG + SlotGCGDefense=Smooth LLM (swap)2026.06 | 96 | |
| GCG-Hij + SlotGCGDefense=Smooth LLM (swap)2026.06 | 96 | |
| I-GCGModel=Vicuna-7B, Attack Strategy=I-GCG2026.06 | 96 | |
| SlotGCG + I-GCGModel=Vicuna-7B, Attack Strategy=I-GCG2026.06 | 96 | |
| EnumAttackModel=gemini-2.0-flash, Grammar Guardrail=None2025.03 | 95.8 | |
| TemplateFuzzModel=Llama2-7B, Attack Variant=Top-1 Template2026.04 | 95.58 | |
| DictAttackModel=gpt-4o-mini, Grammar Guardrail=llama-guard-3-8b, k=12025.03 | 95.5 | |
| DictAttackModel=gpt-5, Grammar Guardrail=enabled2025.03 | 94.4 | |
| SlotGCG + I-GCGModel=Llama-2-13B, Attack Strategy=I-GCG2026.06 | 94 | |
| AttnGCGModel=Mistral-7B, Attack Strategy=AttnGCG2026.06 | 94 | |
| ICD-PREFILLTarget Model=Vicuna-1.5-13B2026.04 | 93.08 | |
| AmpleGCGTarget Model=Llama-2-7b, Suffix Length=202025.09 | 93 | |
| TemplateFuzzModel=Llama3-8B, Attack Variant=Top-1 Template2026.04 | 92.16 | |
| AttnGCG + SlotGCGDefense=Smooth LLM (swap)2026.06 | 92 | |
| SlotGCG + AttnGCGModel=Mistral-7B, Attack Strategy=AttnGCG2026.06 | 92 | |
| TemplateFuzzModel=Qwen3-8B, Attack Variant=Top-1 Template2026.04 | 91.54 | |
| TemplateFuzzModel=Qwen2.5-7B, Attack Variant=Top-1 Template2026.04 | 90.96 | |
| SlotGCG + AttnGCGModel=Llama-2-7B, Attack Strategy=AttnGCG2026.06 | 90 | |
| SlotGCG + I-GCGModel=Llama-2-7B, Attack Strategy=I-GCG2026.06 | 90 | |
| SlotGCG + GCG-HijModel=Llama-2-7B, Attack Strategy=GCG-Hij2026.06 | 90 | |
| SlotGCG + GCG-HijModel=Llama-2-13B, Attack Strategy=GCG-Hij2026.06 | 90 | |
| I-GCGModel=Mistral-7B, Attack Strategy=I-GCG2026.06 | 90 | |
| SlotGCG + I-GCGModel=Mistral-7B, Attack Strategy=I-GCG2026.06 | 90 | |
| AttnGCGModel=Vicuna-7B, Attack Strategy=AttnGCG2026.06 | 88 | |
| SlotGCG + AttnGCGModel=Average, Attack Strategy=AttnGCG2026.06 | 86.3 | |
| GCG + SlotGCGDefense=Smooth LLM (swap)2026.06 | 86 | |
| GCGModel=Mistral-7B, Attack Strategy=GCG2026.06 | 86 | |
| SlotGCG + GCGModel=Mistral-7B, Attack Strategy=GCG2026.06 | 86 | |
| SlotGCG + GCGModel=Vicuna-7B, Attack Strategy=GCG2026.06 | 86 | |
| GCG-HijModel=Vicuna-7B, Attack Strategy=GCG-Hij2026.06 | 86 | |
| SlotGCG + GCG-HijModel=Qwen-2.5, Attack Strategy=GCG-Hij2026.06 | 86 | |
| SlotGCG + I-GCGModel=Average, Attack Strategy=I-GCG2026.06 | 85.7 | |
| SlotGCG + GCG-HijModel=Average, Attack Strategy=GCG-Hij2026.06 | 84.3 | |
| GCG-HijModel=Mistral-7B, Attack Strategy=GCG-Hij2026.06 | 84 | |
| ICD-PREFILLTarget Model=Llama-3.1-70B2026.04 | 82.88 | |
| I-GCG + SlotGCGDefense=Erase-and-Check (infusion)2026.06 | 82 | |
| I-GCG + SlotGCGDefense=SmoothLLM (insert)2026.06 | 82 | |
| SlotGCG + AttnGCGModel=Llama-2-13B, Attack Strategy=AttnGCG2026.06 | 82 | |
| SlotGCG + GCGModel=Llama-3.1-8B, Attack Strategy=GCG2026.06 | 82 | |
| SlotGCG + AttnGCGModel=Llama-3.1-8B, Attack Strategy=AttnGCG2026.06 | 82 | |
| SlotGCG + AttnGCGModel=Vicuna-7B, Attack Strategy=AttnGCG2026.06 | 82 | |
| SlotGCG + GCG-HijModel=Vicuna-7B, Attack Strategy=GCG-Hij2026.06 | 82 | |
| GCG-HijModel=Qwen-2.5, Attack Strategy=GCG-Hij2026.06 | 82 | |
| GCGTarget Model=Average, Suffix Length=302025.09 | 81 | |
| GCGTarget Model=Llama-2-13b, Suffix Length=302025.09 | 80 | |
| I-GCG + SlotGCGDefense=SmoothLLM (patch)2026.06 | 80 | |
| SlotGCG + GCGModel=Llama-2-7B, Attack Strategy=GCG2026.06 | 80 | |
| SlotGCG + GCG-HijModel=Mistral-7B, Attack Strategy=GCG-Hij2026.06 | 80 |