Jailbreak Attack on AdvBench 50 harmful behaviors
100GPT-3.5 Turbo Jailbreak RatePromptAttack
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| PromptAttackAttack Type=Black-box Attack Method2024.10 | 100 | 100 | 100 | 100 | 36 | 24 | 38 | 16 | 64.25 | |
| ArtPromptAttack Type=Black-box Attack Method2024.10 | 98 | 64 | 74 | 82 | 76 | 18 | 12 | 58 | 60.25 | |
| CodeChameleonAttack Type=Black-box Attack Method2024.10 | 98 | 94 | 32 | 92 | 84 | 44 | 0 | 60 | 63 | |
| ReNeLLMAttack Type=Black-box Attack Method2024.10 | 94 | 84 | 72 | 92 | 86 | 18 | 10 | 62 | 64.75 | |
| Base64Attack Type=Black-box Attack Method2024.10 | 92 | 94 | 84 | 98 | 74 | 100 | 0 | 94 | 79.5 | |
| CodeChameleonAttack Method Type=Black-box Attack Method2024.10 | 92 | 100 | 28 | 98 | 62 | 22 | 0 | 92 | 61.75 | |
| ReNeLLMAttack Method Type=Black-box Attack Method2024.10 | 92 | 88 | 60 | 86 | 50 | 4 | 2 | 54 | 54.5 | |
| FlipAttackAttack Method Type=Black-box Attack Method2024.10 | 90 | 92 | 86 | 90 | 56 | 84 | 24 | 94 | 77 | |
| AutoDANAttack Method Type=White-box Attack Method2024.10 | 86 | 28 | 16 | 42 | 28 | 0 | 0 | 76 | 34.5 | |
| FlipAttackAttack Type=Black-box Attack Method2024.10 | 80 | 82 | 68 | 70 | 88 | 92 | 78 | 52 | 76.25 | |
| PAIRAttack Type=Black-box Attack Method2024.10 | 78 | 48 | 36 | 50 | 12 | 20 | 18 | 18 | 35 | |
| TAPAttack Type=Black-box Attack Method2024.10 | 78 | 66 | 64 | 46 | 8 | 12 | 16 | 16 | 38.25 | |
| DeepInceptionAttack Type=Black-box Attack Method2024.10 | 76 | 78 | 62 | 80 | 68 | 16 | 12 | 90 | 60.25 | |
| PAIRAttack Method Type=Black-box Attack Method2024.10 | 70 | 32 | 36 | 44 | 4 | 0 | 6 | 6 | 24.75 | |
| AutoDANAttack Type=White-box Attack Method2024.10 | 68 | 22 | 14 | 40 | 26 | 18 | 6 | 66 | 32.5 | |
| TAPAttack Method Type=Black-box Attack Method2024.10 | 64 | 34 | 42 | 60 | 10 | 0 | 4 | 38 | 31.5 | |
| MACAttack Method Type=White-box Attack Method2024.10 | 50 | 0 | 0 | 0 | 4 | 0 | 0 | 20 | 9.25 | |
| GCGAttack Method Type=White-box Attack Method2024.10 | 38 | 0 | 2 | 0 | 4 | 0 | 0 | 18 | 7.75 | |
| DeepInceptionAttack Method Type=Black-box Attack Method2024.10 | 38 | 8 | 30 | 40 | 20 | 0 | 0 | 46 | 22.75 | |
| COLD-AttackAttack Method Type=White-box Attack Method2024.10 | 36 | 0 | 0 | 0 | 4 | 0 | 0 | 14 | 6.75 | |
| Base64Attack Method Type=Black-box Attack Method2024.10 | 36 | 0 | 0 | 64 | 4 | 0 | 0 | 4 | 13.5 | |
| GCGAttack Type=White-box Attack Method2024.10 | 32 | 2 | 2 | 0 | 4 | 12 | 0 | 8 | 7.5 | |
| GPTFuzzerAttack Type=Black-box Attack Method2024.10 | 32 | 46 | 24 | 50 | 28 | 24 | 0 | 42 | 30.75 | |
| GPTFuzzerAttack Method Type=Black-box Attack Method2024.10 | 26 | 46 | 34 | 70 | 34 | 0 | 0 | 70 | 35 | |
| PromptAttackAttack Method Type=Black-box Attack Method2024.10 | 24 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 3 | |
| COLD-AttackAttack Type=White-box Attack Method2024.10 | 22 | 0 | 0 | 0 | 0 | 14 | 0 | 2 | 4.75 | |
| MACAttack Type=White-box Attack Method2024.10 | 20 | 0 | 0 | 0 | 4 | 8 | 0 | 4 | 4.5 | |
| ArtPromptAttack Method Type=Black-box Attack Method2024.10 | 20 | 6 | 2 | 0 | 0 | 0 | 0 | 20 | 6 | |
| DRAAttack Type=Black-box Attack Method2024.10 | 4 | 22 | 28 | 30 | 2 | 6 | 0 | 58 | 18.75 | |
| DRAAttack Method Type=Black-box Attack Method2024.10 | 4 | 12 | 24 | 36 | 0 | 0 | 0 | 62 | 17.25 | |
| SelfCipherAttack Type=Black-box Attack Method2024.10 | 2 | 0 | 68 | 0 | 22 | 10 | 0 | 2 | 13 | |
| SelfCipherAttack Method Type=Black-box Attack Method2024.10 | 0 | 0 | 36 | 0 | 0 | 0 | 0 | 0 | 4.5 |