Jailbreaking on HarmBench Transfer attack
58.7Average Success RateAttention-Guided Visual Jailbreaking
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Attention-Guided Visual JailbreakingTarget Model=Gemini-1.5, Source Model=Qwen-VL, Perturbation budget (epsilon)=32/2552026.04 | 58.7 | — | — | — | — | — | — | |
| Vicuna-13BDefense=None2024.01 | 48 | 65.6 | 65.9 | 50.3 | 53.6 | 32.2 | 20.1 | |
| Attention-Guided Visual JailbreakingTarget Model=Claude-3.5, Source Model=Qwen-VL, Perturbation budget (epsilon)=32/2552026.04 | 41.5 | — | — | — | — | — | — | |
| RPOBase Model=Vicuna-13B, Suffix Length=20 tokens2024.01 | 29.6 | 17.8 | 59.5 | 32.5 | 37.2 | 13 | 17.7 | |
| GPT-3.5Defense=None2024.01 | 27.2 | 42.6 | 6.5 | 36.3 | 38.9 | 27.6 | 11.3 | |
| GPT-4Defense=None2024.01 | 19.2 | 22.3 | 0.5 | 33.8 | 37.6 | 9.3 | 11.6 | |
| RPOBase Model=GPT-3.5, Suffix Length=20 tokens2024.01 | 18.5 | 9.3 | 3.2 | 29.4 | 33 | 25.9 | 10 | |
| Attention-Guided Visual JailbreakingTarget Model=GPT-4o, Source Model=Qwen-VL, Perturbation budget (epsilon)=32/2552026.04 | 15.8 | — | — | — | — | — | — | |
| RPOBase Model=GPT-4, Suffix Length=20 tokens2024.01 | 15.7 | 9 | 0.2 | 31.2 | 35.8 | 7 | 10.9 | |
| VAETarget Model=Gemini-1.5, Source Model=Qwen-VL, Perturbation budget (epsilon)=32/2552026.04 | 13.2 | — | — | — | — | — | — | |
| VAETarget Model=GPT-4o, Source Model=Qwen-VL, Perturbation budget (epsilon)=32/2552026.04 | 11.5 | — | — | — | — | — | — | |
| Llama-2-7BDefense=None2024.01 | 9.7 | 31.9 | 0 | 9.4 | 9.1 | 5 | 2.7 | |
| VAETarget Model=Claude-3.5, Source Model=Qwen-VL, Perturbation budget (epsilon)=32/2552026.04 | 7.9 | — | — | — | — | — | — | |
| RPOBase Model=Llama-2-7B, Suffix Length=20 tokens2024.01 | 3.2 | 6.7 | 0 | 5 | 7.8 | 0 | 0 |