Jailbreak Attack on AdvBench (ASR, MSS, ASS, LFS)
100ASREmbedding Attack (Standard)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Embedding Attack (Standard)Attack Algorithm=Embedding Attack, Initialization=Standard, Target Model=Llama-22025.02 | 100 | 12 | 12.5 | 1.71 | |
| Embedding Attack (Baseline)Attack Algorithm=Embedding Attack, Initialization=Baseline, Target Model=Llama-22025.02 | 100 | 3 | 3.72 | 1.16 | |
| Embedding Attack (25-CRI)Attack Algorithm=Embedding Attack, Initialization=25-CRI, Target Model=Llama-22025.02 | 100 | 1 | 1.41 | 0.08 | |
| Embedding Attack (5-CRI)Attack Algorithm=Embedding Attack, Initialization=5-CRI, Target Model=Llama-22025.02 | 100 | 2 | 2.6 | 0.51 | |
| Embedding Attack (1-CRI)Attack Algorithm=Embedding Attack, Initialization=1-CRI, Target Model=Llama-22025.02 | 100 | 7 | 6.84 | 3.72 | |
| Embedding Attack (Baseline)Attack Algorithm=Embedding Attack, Initialization=Baseline, Target Model=Vicuna2025.02 | 100 | 3 | 3.96 | 0.95 | |
| Embedding Attack (5-CRI)Attack Algorithm=Embedding Attack, Initialization=5-CRI, Target Model=Vicuna2025.02 | 100 | 2 | 2.58 | 0.51 | |
| AutoDAN-GA (Standard)Attack Algorithm=AutoDAN-GA, Initialization=Standard, Target Model=Vicuna2025.02 | 100 | 2 | 2.21 | 0.84 | |
| AutoDAN-GA (25-CRI)Attack Algorithm=AutoDAN-GA, Initialization=25-CRI, Target Model=Vicuna2025.02 | 100 | 2 | 2.2 | 0.65 | |
| AutoDAN-HGA (Standard)Attack Algorithm=AutoDAN-HGA, Initialization=Standard, Target Model=Vicuna2025.02 | 100 | 2 | 2.32 | 0.84 | |
| AutoDAN-HGA (25-CRI)Attack Algorithm=AutoDAN-HGA, Initialization=25-CRI, Target Model=Vicuna2025.02 | 100 | 2 | 2.38 | 0.37 | |
| GCG (5-CRI)Attack Algorithm=GCG, Initialization=5-CRI, Target Model=Llama-32025.02 | 100 | 1 | 11.29 | 0.63 | |
| GCG (1-CRI)Attack Algorithm=GCG, Initialization=1-CRI, Target Model=Llama-32025.02 | 100 | 1 | 15.12 | 1 | |
| Embedding Attack (25-CRI)Attack Algorithm=Embedding Attack, Initialization=25-CRI, Target Model=Llama-32025.02 | 100 | 7 | 8.56 | 4.36 | |
| Embedding Attack (5-CRI)Attack Algorithm=Embedding Attack, Initialization=5-CRI, Target Model=Llama-32025.02 | 100 | 2 | 2.44 | 0.6 | |
| Embedding Attack (1-CRI)Attack Algorithm=Embedding Attack, Initialization=1-CRI, Target Model=Llama-32025.02 | 100 | 6 | 6.02 | 5.77 | |
| AutoDAN-GA (Standard)Attack Algorithm=AutoDAN-GA, Initialization=Standard, Target Model=Llama-32025.02 | 100 | 3 | 5.67 | 1.58 | |
| AutoDAN-GA (25-CRI)Attack Algorithm=AutoDAN-GA, Initialization=25-CRI, Target Model=Llama-32025.02 | 100 | 3 | 3.98 | 1.17 | |
| AutoDAN-HGA (Standard)Attack Algorithm=AutoDAN-HGA, Initialization=Standard, Target Model=Llama-32025.02 | 100 | 2 | 7.48 | 1.58 | |
| AutoDAN-HGA (25-CRI)Attack Algorithm=AutoDAN-HGA, Initialization=25-CRI, Target Model=Llama-32025.02 | 100 | 2 | 5.94 | 1.03 | |
| GCG (1-CRI)Attack Algorithm=GCG, Initialization=1-CRI, Target Model=Llama-22025.02 | 99 | 1 | 1.68 | 0.33 | |
| GCG (Random)Attack Algorithm=GCG, Initialization=Random, Target Model=Vicuna2025.02 | 98 | 9 | 15.8 | 0.58 | |
| GCG (1-CRI)Attack Algorithm=GCG, Initialization=1-CRI, Target Model=Vicuna2025.02 | 98 | 1 | 1.01 | 0.25 | |
| GCG (25-CRI)Attack Algorithm=GCG, Initialization=25-CRI, Target Model=Llama-32025.02 | 98 | 4 | 18.4 | 0.74 | |
| Embedding Attack (Standard)Attack Algorithm=Embedding Attack, Initialization=Standard, Target Model=Llama-32025.02 | 98 | 22 | 28 | 3.3 | |
| Embedding Attack (Baseline)Attack Algorithm=Embedding Attack, Initialization=Baseline, Target Model=Llama-32025.02 | 98 | 4 | 6.36 | 2.12 | |
| GCG (25-CRI)Attack Algorithm=GCG, Initialization=25-CRI, Target Model=Llama-22025.02 | 97 | 1 | 2.04 | 0.21 | |
| GCG (5-CRI)Attack Algorithm=GCG, Initialization=5-CRI, Target Model=Llama-22025.02 | 97 | 1 | 1.2 | 0.28 | |
| GCG (Standard)Attack Algorithm=GCG, Initialization=Standard, Target Model=Vicuna2025.02 | 97 | 8 | 15.75 | 0.8 | |
| GCG (25-CRI)Attack Algorithm=GCG, Initialization=25-CRI, Target Model=Vicuna2025.02 | 97 | 1 | 2.83 | 0.23 | |
| GCG (5-CRI)Attack Algorithm=GCG, Initialization=5-CRI, Target Model=Vicuna2025.02 | 97 | 1 | 1.61 | 0.31 | |
| AutoDAN-HGA (25-CRI)Attack Algorithm=AutoDAN-HGA, Initialization=25-CRI, Target Model=Llama-22025.02 | 92 | 4.5 | 13.8 | 1.48 | |
| GCG (Standard)Attack Algorithm=GCG, Initialization=Standard, Target Model=Llama-22025.02 | 90 | 64 | 106.51 | 2.29 | |
| GCG (Standard)Attack Algorithm=GCG, Initialization=Standard, Target Model=Llama-32025.02 | 89 | 76 | 118.27 | 1.84 | |
| Embedding Attack (Standard)Attack Algorithm=Embedding Attack, Initialization=Standard, Target Model=Vicuna2025.02 | 86 | 27 | 36.38 | 2.99 | |
| Embedding Attack (25-CRI)Attack Algorithm=Embedding Attack, Initialization=25-CRI, Target Model=Vicuna2025.02 | 86 | 9 | 21.58 | 2.29 | |
| Embedding Attack (1-CRI)Attack Algorithm=Embedding Attack, Initialization=1-CRI, Target Model=Vicuna2025.02 | 86 | 9 | 21.26 | 3.63 | |
| GCG (Random)Attack Algorithm=GCG, Initialization=Random, Target Model=Llama-32025.02 | 78 | 89 | 132.26 | 1.76 | |
| GCG (Random)Attack Algorithm=GCG, Initialization=Random, Target Model=Llama-22025.02 | 76 | 80 | 114.93 | 1.9 | |
| AutoDAN-HGA (Standard)Attack Algorithm=AutoDAN-HGA, Initialization=Standard, Target Model=Llama-22025.02 | 67 | 19.5 | 29.6 | 2.25 | |
| AutoDAN-GA (25-CRI)Attack Algorithm=AutoDAN-GA, Initialization=25-CRI, Target Model=Llama-22025.02 | 30 | 8 | 9.55 | 1.9 | |
| AutoDAN-GA (Standard)Attack Algorithm=AutoDAN-GA, Initialization=Standard, Target Model=Llama-22025.02 | 19 | 9 | 9.84 | 2.25 |