Jailbreak on AdvBench (Intra-episode)
100ASRAutoDAN (loss)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| AutoDAN (loss)Paradigm=iterative w/o tools, Target Model=LLaMA2026.06 | 100 | 8.25 | |
| TAPParadigm=iterative with tools, Target Model=Qwen32026.06 | 100 | 11.77 | |
| AutoDAN (loss)Paradigm=iterative with tools, Target Model=LLaMA2026.06 | 100 | 8 | |
| AutoDAN (LLM-judge)Paradigm=iterative with tools, Target Model=Qwen32026.06 | 100 | 8.89 | |
| AutoDAN (LLM-judge)Paradigm=iterative with tools, Target Model=DeepSeek2026.06 | 100 | 8.03 | |
| TAPParadigm=iterative with tools, Target Model=Vicuna2026.06 | 99.81 | 9.28 | |
| AutoDAN (LLM-judge)Paradigm=iterative with tools, Target Model=GPT-4o2026.06 | 99.81 | 8.37 | |
| TAPParadigm=iterative w/o tools, Target Model=Vicuna2026.06 | 99.62 | 8.8 | |
| AutoDAN (LLM-judge)Paradigm=iterative w/o tools, Target Model=Vicuna2026.06 | 99.62 | 8 | |
| TAPParadigm=iterative with tools, Target Model=GPT-4o2026.06 | 99.62 | 9.74 | |
| TAPParadigm=iterative with tools, Target Model=DeepSeek2026.06 | 99.62 | 8.5 | |
| flipParadigm=single-turn attack tool, Target Model=DeepSeek2026.06 | 99.23 | 1 | |
| AutoDAN (LLM-judge)Paradigm=iterative with tools, Target Model=Vicuna2026.06 | 99.23 | 14.06 | |
| AutoDAN (loss)Paradigm=iterative with tools, Target Model=Vicuna2026.06 | 98.46 | 8.5 | |
| AutoDAN (loss)Paradigm=iterative w/o tools, Target Model=Vicuna2026.06 | 98.27 | 9.57 | |
| TAPParadigm=iterative w/o tools, Target Model=DeepSeek2026.06 | 98.08 | 11.85 | |
| AutoDAN (LLM-judge)Paradigm=iterative with tools, Target Model=LLaMA2026.06 | 97.69 | 14.57 | |
| TAPParadigm=iterative with tools, Target Model=LLaMA2026.06 | 97.56 | 17.2 | |
| TAPParadigm=iterative w/o tools, Target Model=LLaMA2026.06 | 95.77 | 15.48 | |
| GCGParadigm=iterative w/o tools, Target Model=Vicuna2026.06 | 94 | 64 | |
| GCGParadigm=iterative with tools, Target Model=Vicuna2026.06 | 94 | 1.57 | |
| code chameleonParadigm=single-turn attack tool, Target Model=DeepSeek2026.06 | 93.85 | 1 | |
| AutoDAN (LLM-judge)Paradigm=iterative w/o tools, Target Model=GPT-4o2026.06 | 93.27 | 8.54 | |
| AutoDAN (LLM-judge)Paradigm=iterative w/o tools, Target Model=LLaMA2026.06 | 93.08 | 9.34 | |
| GCGParadigm=iterative w/o tools, Target Model=LLaMA2026.06 | 92 | 64 | |
| AutoDAN (LLM-judge)Paradigm=iterative w/o tools, Target Model=DeepSeek2026.06 | 91.35 | 9.31 | |
| AutoDAN (LLM-judge)Paradigm=iterative with tools, Target Model=GPT-5.12026.06 | 91.15 | 13.87 | |
| GCGParadigm=iterative with tools, Target Model=LLaMA2026.06 | 90 | 1.27 | |
| TAPParadigm=iterative w/o tools, Target Model=GPT-4o2026.06 | 88.75 | 15.82 | |
| TAPParadigm=iterative with tools, Target Model=GPT-5.12026.06 | 83.46 | 13.86 | |
| flipParadigm=single-turn attack tool, Target Model=GPT-4o2026.06 | 82.31 | 1 | |
| TAPParadigm=iterative w/o tools, Target Model=Qwen32026.06 | 80.96 | 17.84 | |
| AutoDAN (LLM-judge)Paradigm=iterative with tools, Target Model=Gemini2026.06 | 76.73 | 22.96 | |
| AutoDAN (LLM-judge)Paradigm=iterative w/o tools, Target Model=Gemini2026.06 | 69.81 | 9.74 | |
| code chameleonParadigm=single-turn attack tool, Target Model=Gemini2026.06 | 63.65 | 1 | |
| code chameleonParadigm=single-turn attack tool, Target Model=Qwen32026.06 | 63.08 | 1 | |
| flipParadigm=single-turn attack tool, Target Model=LLaMA2026.06 | 62.69 | 1 | |
| AutoDAN (LLM-judge)Paradigm=iterative w/o tools, Target Model=GPT-5.12026.06 | 60.19 | 17.35 | |
| AutoDAN (LLM-judge)Paradigm=iterative with tools, Target Model=Claude2026.06 | 59.81 | 9.98 | |
| flipParadigm=single-turn attack tool, Target Model=Qwen32026.06 | 56.73 | 1 | |
| TAPParadigm=iterative with tools, Target Model=Claude2026.06 | 56.54 | 23.37 | |
| code chameleonParadigm=single-turn attack tool, Target Model=Claude2026.06 | 40.77 | 1 | |
| code chameleonParadigm=single-turn attack tool, Target Model=GPT-4o2026.06 | 37.5 | 1 | |
| TAPParadigm=iterative w/o tools, Target Model=GPT-5.12026.06 | 29.46 | 29.01 | |
| flipParadigm=single-turn attack tool, Target Model=Gemini2026.06 | 28.08 | 1 | |
| persuasionParadigm=single-turn attack tool, Target Model=Vicuna2026.06 | 26.73 | 1 | |
| code chameleonParadigm=single-turn attack tool, Target Model=Vicuna2026.06 | 24.42 | 1 | |
| code chameleonParadigm=single-turn attack tool, Target Model=GPT-5.12026.06 | 23.08 | 1 | |
| code chameleonParadigm=single-turn attack tool, Target Model=LLaMA2026.06 | 22.31 | 1 | |
| TAPParadigm=iterative with tools, Target Model=Gemini2026.06 | 20.19 | 34.67 | |
| AutoDAN (LLM-judge)Paradigm=iterative w/o tools, Target Model=Qwen32026.06 | 16.15 | 16.95 | |
| TAPParadigm=iterative w/o tools, Target Model=Gemini2026.06 | 9.79 | 31.37 | |
| persuasionParadigm=single-turn attack tool, Target Model=LLaMA2026.06 | 4.62 | 1 | |
| persuasionParadigm=single-turn attack tool, Target Model=GPT-4o2026.06 | 3.85 | 1 | |
| flipParadigm=single-turn attack tool, Target Model=Vicuna2026.06 | 2.31 | 1 | |
| persuasionParadigm=single-turn attack tool, Target Model=DeepSeek2026.06 | 1.92 | 1 | |
| flipParadigm=single-turn attack tool, Target Model=GPT-5.12026.06 | 1.54 | 1 | |
| persuasionParadigm=single-turn attack tool, Target Model=Gemini2026.06 | 1.35 | 1 | |
| TAPParadigm=iterative w/o tools, Target Model=Claude2026.06 | 0.96 | 25.4 | |
| persuasionParadigm=single-turn attack tool, Target Model=GPT-5.12026.06 | 0.77 | 1 | |
| persuasionParadigm=single-turn attack tool, Target Model=Qwen32026.06 | 0.77 | 1 | |
| AutoDAN (LLM-judge)Paradigm=iterative w/o tools, Target Model=Claude2026.06 | 0.38 | 8 | |
| flipParadigm=single-turn attack tool, Target Model=Claude2026.06 | 0 | 1 | |
| persuasionParadigm=single-turn attack tool, Target Model=Claude2026.06 | 0 | 1 |