Jailbreak Attack on AdvBench 150 Harmful Behaviors
0ASRRA-LLM
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| RA-LLMAttack=Adv Strings, Model Backbone=Vicuna-7B-v1.32023.09 | 0 | 84 | |
| TAPTarget Model=Llama-2-7B-chat2026.03 | 4 | — | |
| PAIRTarget Model=Llama-2-7B-chat2026.03 | 10 | — | |
| RA-LLMAttack Type=TAP, Base Model=Guanaco-7B-HF2023.09 | 12.7 | 85.3 | |
| RA-LLMAttack Type=AutoDAN-GPT, Base Model=Guanaco-7B-HF2023.09 | 15.3 | 84.7 | |
| RA-LLMAttack Type=TAP, Base Model=Vicuna-7B-chat-HF2023.09 | 16 | 82 | |
| RA-LLMAttack Type=AutoDAN, Base Model=Guanaco-7B-HF2023.09 | 16.7 | 82 | |
| AdvPrompterTarget Model=Llama-2-7B-chat2026.03 | 24 | — | |
| AutoDANTarget Model=Llama-2-7B-chat2026.03 | 26 | — | |
| AmpleGCGTarget Model=Llama-2-7B-chat2026.03 | 28 | — | |
| Baseline2026.03 | 34.8 | — | |
| RA-LLMAttack Type=AutoDAN-GPT, Base Model=Vicuna-7B-chat-HF2023.09 | 41.3 | 47.4 | |
| RA-LLMAttack Type=AutoDAN, Base Model=Vicuna-7B-chat-HF2023.09 | 42 | 48 | |
| GCGTarget Model=Llama-2-7B-chat2026.03 | 54 | — | |
| MACTarget Model=Llama-2-7B-chat2026.03 | 56 | — | |
| Probe-SamplingTarget Model=Llama-2-7B-chat2026.03 | 56 | — | |
| AdvPrompterTarget Model=Vicuna-7B-1.52026.03 | 64 | — | |
| AmpleGCGTarget Model=Vicuna-7B-1.52026.03 | 66 | — | |
| AdvPrompterTarget Model=Mistral-7B-Instruct-0.22026.03 | 74 | — | |
| Original LLMAttack=Adv Strings, Model Backbone=Vicuna-7B-v1.32023.09 | 84 | — | |
| Amnesia2026.03 | 86.3 | 51.5 | |
| Original LLMAttack Type=AutoDAN-GPT, Base Model=Vicuna-7B-chat-HF2023.09 | 88.7 | — | |
| PAIRTarget Model=Mistral-7B-Instruct-0.22026.03 | 90 | — | |
| Original LLMAttack Type=AutoDAN, Base Model=Vicuna-7B-chat-HF2023.09 | 90.7 | — | |
| GCGTarget Model=Mistral-7B-Instruct-0.22026.03 | 92 | — | |
| TAPTarget Model=Mistral-7B-Instruct-0.22026.03 | 92 | — | |
| MACTarget Model=Mistral-7B-Instruct-0.22026.03 | 94 | — | |
| Probe-SamplingTarget Model=Mistral-7B-Instruct-0.22026.03 | 94 | — | |
| PAIRTarget Model=Vicuna-7B-1.52026.03 | 94 | — | |
| TAPTarget Model=Vicuna-7B-1.52026.03 | 94 | — | |
| AutoDANTarget Model=Mistral-7B-Instruct-0.22026.03 | 96 | — | |
| Original LLMAttack Type=TAP, Base Model=Vicuna-7B-chat-HF2023.09 | 98 | — | |
| Original LLMAttack Type=TAP, Base Model=Guanaco-7B-HF2023.09 | 98 | — | |
| GCGTarget Model=Vicuna-7B-1.52026.03 | 98 | — | |
| Original LLMAttack Type=AutoDAN, Base Model=Guanaco-7B-HF2023.09 | 98.7 | — | |
| Original LLMAttack Type=AutoDAN-GPT, Base Model=Guanaco-7B-HF2023.09 | 100 | — | |
| MACTarget Model=Vicuna-7B-1.52026.03 | 100 | — | |
| AutoDANTarget Model=Vicuna-7B-1.52026.03 | 100 | — | |
| Probe-SamplingTarget Model=Vicuna-7B-1.52026.03 | 100 | — | |
| I-GCGTarget Model=Vicuna-7B-1.52026.03 | 100 | — | |
| I-GCGTarget Model=Llama-2-7B-chat2026.03 | 100 | — | |
| I-GCGTarget Model=Mistral-7B-Instruct-0.22026.03 | 100 | — | |
| TAO-AttackTarget Model=Vicuna-7B-1.52026.03 | 100 | — | |
| TAO-AttackTarget Model=Llama-2-7B-chat2026.03 | 100 | — | |
| TAO-AttackTarget Model=Mistral-7B-Instruct-0.22026.03 | 100 | — |