Jailbreak Attack on AdvBench 50
100ASR (KW)FlipAttack
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| FlipAttackVictim Model=DeepSeek-V3.2, Search Setting=w/ snippet2026.01 | 100 | 92 | — | — | — | — | — | — | — | — | — | |
| SearchAttackVictim Model=DeepSeek-V3.2, Search Setting=w/ snippet2026.01 | 100 | 96 | — | — | — | — | — | — | — | — | — | |
| SearchAttackVictim Model=Qwen3-32B, Search Setting=w/o search2026.01 | 100 | 94 | — | — | — | — | — | — | — | — | — | |
| FlipAttackVictim Model=DeepSeek-V3.2, Search Setting=w/o search2026.01 | 98 | 92 | — | — | — | — | — | — | — | — | — | |
| FlipAttackVictim Model=DeepSeek-V3.2, Search Setting=w/ agentic2026.01 | 98 | 96 | — | — | — | — | — | — | — | — | — | |
| SearchAttackVictim Model=Qwen3-32B, Search Setting=w/ snippet2026.01 | 98 | 94 | — | — | — | — | — | — | — | — | — | |
| SearchAttackVictim Model=Qwen3-32B, Search Setting=w/ agentic2026.01 | 98 | 96 | — | — | — | — | — | — | — | — | — | |
| SearchAttackVictim Model=Gemini-3-Flash, Search Setting=w/o search2026.01 | 96 | 96 | — | — | — | — | — | — | — | — | — | |
| SearchAttackVictim Model=DeepSeek-V3.2, Search Setting=w/o search2026.01 | 96 | 94 | — | — | — | — | — | — | — | — | — | |
| SearchAttackVictim Model=Gemini-3-Flash, Search Setting=w/ snippet2026.01 | 94 | 96 | — | — | — | — | — | — | — | — | — | |
| ReNeLLMVictim Model=DeepSeek-V3.2, Search Setting=w/ agentic2026.01 | 94 | 68 | — | — | — | — | — | — | — | — | — | |
| SearchAttackVictim Model=DeepSeek-V3.2, Search Setting=w/ agentic2026.01 | 94 | 100 | — | — | — | — | — | — | — | — | — | |
| PAIRVictim Model=Qwen3-32B, Search Setting=w/ agentic2026.01 | 94 | 52 | — | — | — | — | — | — | — | — | — | |
| SearchAttackVictim Model=GPT-4o, Search Setting=w/o search2026.01 | 94 | 92 | — | — | — | — | — | — | — | — | — | |
| SearchAttackVictim Model=GPT-4o, Search Setting=w/ snippet2026.01 | 94 | 94 | — | — | — | — | — | — | — | — | — | |
| PAIRVictim Model=DeepSeek-V3.2, Search Setting=w/ agentic2026.01 | 92 | 42 | — | — | — | — | — | — | — | — | — | |
| PAIRVictim Model=Qwen3-32B, Search Setting=w/o search2026.01 | 92 | 42 | — | — | — | — | — | — | — | — | — | |
| FlipAttackVictim Model=Qwen3-32B, Search Setting=w/o search2026.01 | 92 | 74 | — | — | — | — | — | — | — | — | — | |
| SearchAttackVictim Model=GPT-4o, Search Setting=w/ agentic2026.01 | 92 | 96 | — | — | — | — | — | — | — | — | — | |
| ReNeLLMVictim Model=DeepSeek-V3.2, Search Setting=w/ snippet2026.01 | 88 | 64 | — | — | — | — | — | — | — | — | — | |
| SearchAttackVictim Model=Gemini-3-Flash, Search Setting=w/ agentic2026.01 | 86 | 94 | — | — | — | — | — | — | — | — | — | |
| PAIRVictim Model=Qwen3-32B, Search Setting=w/ snippet2026.01 | 86 | 48 | — | — | — | — | — | — | — | — | — | |
| ReNeLLMVictim Model=GPT-4o, Search Setting=w/o search2026.01 | 86 | 58 | — | — | — | — | — | — | — | — | — | |
| PAIRVictim Model=Gemini-3-Flash, Search Setting=w/ agentic2026.01 | 84 | 52 | — | — | — | — | — | — | — | — | — | |
| PAIRVictim Model=DeepSeek-V3.2, Search Setting=w/ snippet2026.01 | 82 | 36 | — | — | — | — | — | — | — | — | — | |
| ReNeLLMVictim Model=Qwen3-32B, Search Setting=w/ snippet2026.01 | 80 | 54 | — | — | — | — | — | — | — | — | — | |
| FlipAttackVictim Model=Qwen3-32B, Search Setting=w/ agentic2026.01 | 80 | 82 | — | — | — | — | — | — | — | — | — | |
| FlipAttackVictim Model=Gemini-3-Flash, Search Setting=w/o search2026.01 | 76 | 70 | — | — | — | — | — | — | — | — | — | |
| PAIRVictim Model=GPT-4o, Search Setting=w/o search2026.01 | 76 | 36 | — | — | — | — | — | — | — | — | — | |
| ReNeLLMVictim Model=DeepSeek-V3.2, Search Setting=w/o search2026.01 | 74 | 42 | — | — | — | — | — | — | — | — | — | |
| ReNeLLMVictim Model=Qwen3-32B, Search Setting=w/o search2026.01 | 74 | 48 | — | — | — | — | — | — | — | — | — | |
| ReNeLLMVictim Model=Qwen3-32B, Search Setting=w/ agentic2026.01 | 72 | 52 | — | — | — | — | — | — | — | — | — | |
| ReNeLLMVictim Model=Gemini-3-Flash, Search Setting=w/ agentic2026.01 | 66 | 46 | — | — | — | — | — | — | — | — | — | |
| FlipAttackVictim Model=Qwen3-32B, Search Setting=w/ snippet2026.01 | 66 | 78 | — | — | — | — | — | — | — | — | — | |
| PAIRVictim Model=GPT-4o, Search Setting=w/ agentic2026.01 | 66 | 24 | — | — | — | — | — | — | — | — | — | |
| ReNeLLMVictim Model=GPT-4o, Search Setting=w/ snippet2026.01 | 66 | 42 | — | — | — | — | — | — | — | — | — | |
| PAIRVictim Model=Gemini-3-Flash, Search Setting=w/o search2026.01 | 64 | 42 | — | — | — | — | — | — | — | — | — | |
| PAIRVictim Model=DeepSeek-V3.2, Search Setting=w/o search2026.01 | 64 | 28 | — | — | — | — | — | — | — | — | — | |
| PAIRVictim Model=GPT-4o, Search Setting=w/ snippet2026.01 | 64 | 28 | — | — | — | — | — | — | — | — | — | |
| PAIRVictim Model=Gemini-3-Flash, Search Setting=w/ snippet2026.01 | 54 | 46 | — | — | — | — | — | — | — | — | — | |
| ReNeLLMVictim Model=Gemini-3-Flash, Search Setting=w/o search2026.01 | 54 | 24 | — | — | — | — | — | — | — | — | — | |
| ReNeLLMVictim Model=Gemini-3-Flash, Search Setting=w/ snippet2026.01 | 48 | 38 | — | — | — | — | — | — | — | — | — | |
| FlipAttackVictim Model=Gemini-3-Flash, Search Setting=w/ snippet2026.01 | 46 | 42 | — | — | — | — | — | — | — | — | — | |
| ReNeLLMVictim Model=GPT-4o, Search Setting=w/ agentic2026.01 | 34 | 26 | — | — | — | — | — | — | — | — | — | |
| FlipAttackVictim Model=Gemini-3-Flash, Search Setting=w/ agentic2026.01 | 24 | 20 | — | — | — | — | — | — | — | — | — | |
| FlipAttackVictim Model=GPT-4o, Search Setting=w/o search2026.01 | 4 | 2 | — | — | — | — | — | — | — | — | — | |
| FlipAttackVictim Model=GPT-4o, Search Setting=w/ snippet2026.01 | 2 | 2 | — | — | — | — | — | — | — | — | — | |
| FlipAttackVictim Model=GPT-4o, Search Setting=w/ agentic2026.01 | 2 | 2 | — | — | — | — | — | — | — | — | — | |
| AdvWaveTarget Model=Qwen3-Omni2026.05 | — | — | — | — | — | — | 70 | 45 | — | — | — | |
| AdvWaveTarget Model=Qwen2.5-Omni2026.05 | — | — | — | — | — | — | 36 | 4 | — | — | — | |
| AdvWaveTarget Model=LLaMA-Omni2026.05 | — | — | — | — | — | — | 100 | 68 | — | — | — | |
| AutoDAN-Turboevaluation_stage=after Warmup2026.05 | — | — | 84 | — | — | — | — | — | 17.45 | 14 | — | |
| AutoDAN-Turboevaluation_stage=after Warmup2026.05 | — | — | 82 | — | — | — | — | — | 25.9 | 25 | — | |
| DirectTarget Model=Qwen3-Omni2026.05 | — | — | — | — | — | — | 0 | 0 | — | — | — | |
| DirectTarget Model=Qwen2.5-Omni2026.05 | — | — | — | — | — | — | 19 | 1 | — | — | — | |
| DirectTarget Model=LLaMA-Omni2026.05 | — | — | — | — | — | — | 95 | 49 | — | — | — | |
| DirectModel=GPT-4o2026.07 | — | — | 0 | — | — | — | — | — | — | — | — | |
| DirectModel=GPT-4o-mini2026.07 | — | — | 4 | — | — | — | — | — | — | — | — | |
| FigStepTarget Model=Gemini-2.5-Pro2026.02 | — | — | 4 | 0.52 | — | — | — | — | — | — | — | |
| GAPstabilization=global-context2026.05 | — | — | 86 | — | — | — | — | — | 13.09 | 6 | — | |
| GAPstabilization=global-context2026.05 | — | — | 76 | — | — | — | — | — | 16.84 | 13 | — | |
| GPTFuzzerQuery Budget=50, Target Model=Llama-2-7b-chat-hf2026.04 | — | — | — | — | 16 | 9.38 | — | — | — | — | — | |
| GPTFuzzerQuery Budget=100, Target Model=Llama-2-7b-chat-hf2026.04 | — | — | — | — | 30 | 38.7 | — | — | — | — | — | |
| HIMRDTarget Model=Gemini-2.5-Pro2026.02 | — | — | 2 | 0.12 | — | — | — | — | — | — | — | |
| MemoAttackevaluation_stage=after Bootstrap2026.05 | — | — | 96 | — | — | — | — | — | 4.62 | 3 | — | |
| MemoAttackevaluation_stage=after Bootstrap2026.05 | — | — | 98 | — | — | — | — | — | 9.65 | 7 | — | |
| MIDASTarget Model=Gemini-2.5-Pro2026.02 | — | — | 97.96 | 3.9 | — | — | — | — | — | — | — | |
| Past-tenseModel=GPT-4o2026.07 | — | — | 20 | — | — | — | — | — | — | — | — | |
| Past-tenseModel=GPT-4o-mini2026.07 | — | — | 22 | — | — | — | — | — | — | — | — | |
| Post-hoc pruneTarget Model=Qwen3-Omni, ζ=0.252026.05 | — | — | — | — | — | — | 9 | 1 | — | — | — | |
| Post-hoc pruneTarget Model=Qwen2.5-Omni, ζ=0.252026.05 | — | — | — | — | — | — | 38 | 6 | — | — | — | |
| Post-hoc pruneTarget Model=LLaMA-Omni, ζ=0.252026.05 | — | — | — | — | — | — | 97 | 51 | — | — | — | |
| RETROCOTModel=GPT-4o2026.07 | — | — | 58 | — | — | — | — | — | — | — | 44 | |
| RETROCOTModel=GPT-4o-mini2026.07 | — | — | 52 | — | — | — | — | — | — | — | 39 | |
| SpeechGuardTarget Model=Qwen3-Omni2026.05 | — | — | — | — | — | — | 100 | 42 | — | — | — | |
| SpeechGuardTarget Model=Qwen2.5-Omni2026.05 | — | — | — | — | — | — | 94 | 49 | — | — | — | |
| SpeechGuardTarget Model=LLaMA-Omni2026.05 | — | — | — | — | — | — | 100 | 34 | — | — | — | |
| TAGOTarget Model=Qwen3-Omni, ζ=1.02026.05 | — | — | — | — | — | — | 100 | 87 | — | — | — | |
| TAGOTarget Model=Qwen2.5-Omni, ζ=1.02026.05 | — | — | — | — | — | — | 100 | 65 | — | — | — | |
| TAGOTarget Model=LLaMA-Omni, ζ=1.02026.05 | — | — | — | — | — | — | 100 | 71 | — | — | — | |
| TAGOTarget Model=Qwen3-Omni, ζ=0.252026.05 | — | — | — | — | — | — | 99 | 86 | — | — | — | |
| TAGOTarget Model=Qwen2.5-Omni, ζ=0.252026.05 | — | — | — | — | — | — | 97 | 53 | — | — | — | |
| TAGOTarget Model=LLaMA-Omni, ζ=0.252026.05 | — | — | — | — | — | — | 100 | 72 | — | — | — | |
| TAP2026.05 | — | — | 48 | — | — | — | — | — | 27.71 | 29 | — | |
| TAP2026.05 | — | — | 40 | — | — | — | — | — | 33.9 | 33 | — | |
| TemplateFuzzQuery Budget=50, Target Model=Llama-2-7b-chat-hf2026.04 | — | — | — | — | 90 | 9.31 | — | — | — | — | — | |
| TemplateFuzzQuery Budget=100, Target Model=Llama-2-7b-chat-hf2026.04 | — | — | — | — | 94 | 12.89 | — | — | — | — | — | |
| TurboFuzzLLMQuery Budget=50, Target Model=Llama-2-7b-chat-hf2026.04 | — | — | — | — | 40 | 25.05 | — | — | — | — | — | |
| TurboFuzzLLMQuery Budget=100, Target Model=Llama-2-7b-chat-hf2026.04 | — | — | — | — | 66 | 46.12 | — | — | — | — | — |