# Jailbreak Attack on AdvBench 50 harmful behaviors Source: [Jailbreak Attack on AdvBench 50 harmful behaviors]() Collection index: [llms.txt]() Data updated: 2026-05-18T09:28:34.903Z Evaluation of model robustness against jailbreak attacks using the specific 50 harmful behaviors subset derived from AdvBench. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Jailbreak Attack | | Dataset (including any stated split) | AdvBench 50 harmful behaviors | | Metric | GPT-3.5 Turbo Jailbreak Rate | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 32 of 32 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [GPT-3.5 Turbo Jailbreak Rate](): Higher is better; unit: PERCENT. - [GPT-4 Turbo Jailbreak Rate](): Higher is better; unit: PERCENT. - [GPT-4 Jailbreak Rate](): Higher is better; unit: PERCENT. - [GPT-4o Jailbreak Rate](): Higher is better; unit: PERCENT. - [GPT-4o mini Jailbreak Rate](): Higher is better; unit: PERCENT. - [Claude 3.5 Sonnet Jailbreak Rate](): Higher is better; unit: PERCENT. - [LLaMA 3.1 405B Jailbreak Rate](): Higher is better; unit: PERCENT. - [Mixtral 8x22B Jailbreak Rate](): Higher is better; unit: PERCENT. - [Average Jailbreak Rate](): Higher is better; unit: PERCENT. ## Recorded results | Method | GPT-3.5 Turbo Jailbreak Rate | Result date | Report paper | | --- | --- | --- | --- | | PromptAttack | 100 | Not recorded | 68fa751a1e209131cea65ca8 | | ArtPrompt | 98 | Not recorded | 68fa751a1e209131cea65ca8 | | CodeChameleon | 98 | Not recorded | 68fa751a1e209131cea65ca8 | | ReNeLLM | 94 | Not recorded | 68fa751a1e209131cea65ca8 | | Base64 | 92 | Not recorded | 68fa751a1e209131cea65ca8 | | CodeChameleon | 92 | Not recorded | 68fa751a1e209131cea65ca8 | | ReNeLLM | 92 | Not recorded | 68fa751a1e209131cea65ca8 | | FlipAttack | 90 | Not recorded | 68fa751a1e209131cea65ca8 | | AutoDAN | 86 | Not recorded | 68fa751a1e209131cea65ca8 | | FlipAttack | 80 | Not recorded | 68fa751a1e209131cea65ca8 | | PAIR | 78 | Not recorded | 68fa751a1e209131cea65ca8 | | TAP | 78 | Not recorded | 68fa751a1e209131cea65ca8 | | DeepInception | 76 | Not recorded | 68fa751a1e209131cea65ca8 | | PAIR | 70 | Not recorded | 68fa751a1e209131cea65ca8 | | AutoDAN | 68 | Not recorded | 68fa751a1e209131cea65ca8 | | TAP | 64 | Not recorded | 68fa751a1e209131cea65ca8 | | MAC | 50 | Not recorded | 68fa751a1e209131cea65ca8 | | GCG | 38 | Not recorded | 68fa751a1e209131cea65ca8 | | DeepInception | 38 | Not recorded | 68fa751a1e209131cea65ca8 | | COLD-Attack | 36 | Not recorded | 68fa751a1e209131cea65ca8 | | Base64 | 36 | Not recorded | 68fa751a1e209131cea65ca8 | | GCG | 32 | Not recorded | 68fa751a1e209131cea65ca8 | | GPTFuzzer | 32 | Not recorded | 68fa751a1e209131cea65ca8 | | GPTFuzzer | 26 | Not recorded | 68fa751a1e209131cea65ca8 | | PromptAttack | 24 | Not recorded | 68fa751a1e209131cea65ca8 | | COLD-Attack | 22 | Not recorded | 68fa751a1e209131cea65ca8 | | MAC | 20 | Not recorded | 68fa751a1e209131cea65ca8 | | ArtPrompt | 20 | Not recorded | 68fa751a1e209131cea65ca8 | | DRA | 4 | Not recorded | 68fa751a1e209131cea65ca8 | | DRA | 4 | Not recorded | 68fa751a1e209131cea65ca8 | | SelfCipher | 2 | Not recorded | 68fa751a1e209131cea65ca8 | | SelfCipher | 0 | Not recorded | 68fa751a1e209131cea65ca8 | ### PromptAttack GPT-3.5 Turbo Jailbreak Rate: 100. Result ID: 6a0adb6a47f34ce924f14f2f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Type":"Black-box Attack Method"}. This record does not assert independent reproduction. ### ArtPrompt GPT-3.5 Turbo Jailbreak Rate: 98. Result ID: 6a0adb6a47f34ce924f14f2e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Type":"Black-box Attack Method"}. This record does not assert independent reproduction. ### CodeChameleon GPT-3.5 Turbo Jailbreak Rate: 98. Result ID: 6a0adb6a47f34ce924f14f31. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Type":"Black-box Attack Method"}. This record does not assert independent reproduction. ### ReNeLLM GPT-3.5 Turbo Jailbreak Rate: 94. Result ID: 6a0adb6a47f34ce924f14f32. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Type":"Black-box Attack Method"}. This record does not assert independent reproduction. ### Base64 GPT-3.5 Turbo Jailbreak Rate: 92. Result ID: 6a0adb6a47f34ce924f14f2a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Type":"Black-box Attack Method"}. This record does not assert independent reproduction. ### CodeChameleon GPT-3.5 Turbo Jailbreak Rate: 92. Result ID: 6a0adb6a47f34ce924f14f5f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Method Type":"Black-box Attack Method"}. This record does not assert independent reproduction. ### ReNeLLM GPT-3.5 Turbo Jailbreak Rate: 92. Result ID: 6a0adb6a47f34ce924f14f60. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Method Type":"Black-box Attack Method"}. This record does not assert independent reproduction. ### FlipAttack GPT-3.5 Turbo Jailbreak Rate: 90. Result ID: 6a0adb6a47f34ce924f14f61. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Method Type":"Black-box Attack Method"}. This record does not assert independent reproduction. ### AutoDAN GPT-3.5 Turbo Jailbreak Rate: 86. Result ID: 6a0adb6a47f34ce924f14f53. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Method Type":"White-box Attack Method"}. This record does not assert independent reproduction. ### FlipAttack GPT-3.5 Turbo Jailbreak Rate: 80. Result ID: 6a0adb6a47f34ce924f14f33. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Type":"Black-box Attack Method"}. This record does not assert independent reproduction. ### PAIR GPT-3.5 Turbo Jailbreak Rate: 78. Result ID: 6a0adb6a47f34ce924f14f28. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Type":"Black-box Attack Method"}. This record does not assert independent reproduction. ### TAP GPT-3.5 Turbo Jailbreak Rate: 78. Result ID: 6a0adb6a47f34ce924f14f29. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Type":"Black-box Attack Method"}. This record does not assert independent reproduction. ### DeepInception GPT-3.5 Turbo Jailbreak Rate: 76. Result ID: 6a0adb6a47f34ce924f14f2c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Type":"Black-box Attack Method"}. This record does not assert independent reproduction. ### PAIR GPT-3.5 Turbo Jailbreak Rate: 70. Result ID: 6a0adb6a47f34ce924f14f56. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Method Type":"Black-box Attack Method"}. This record does not assert independent reproduction. ### AutoDAN GPT-3.5 Turbo Jailbreak Rate: 68. Result ID: 6a0adb6a47f34ce924f14f25. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Type":"White-box Attack Method"}. This record does not assert independent reproduction. ### TAP GPT-3.5 Turbo Jailbreak Rate: 64. Result ID: 6a0adb6a47f34ce924f14f57. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Method Type":"Black-box Attack Method"}. This record does not assert independent reproduction. ### MAC GPT-3.5 Turbo Jailbreak Rate: 50. Result ID: 6a0adb6a47f34ce924f14f54. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Method Type":"White-box Attack Method"}. This record does not assert independent reproduction. ### GCG GPT-3.5 Turbo Jailbreak Rate: 38. Result ID: 6a0adb6a47f34ce924f14f52. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Method Type":"White-box Attack Method"}. This record does not assert independent reproduction. ### DeepInception GPT-3.5 Turbo Jailbreak Rate: 38. Result ID: 6a0adb6a47f34ce924f14f5a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Method Type":"Black-box Attack Method"}. This record does not assert independent reproduction. ### COLD-Attack GPT-3.5 Turbo Jailbreak Rate: 36. Result ID: 6a0adb6a47f34ce924f14f55. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Attack Method Type":"White-box Attack Method"}. This record does not assert independent reproduction. ### Base64 GPT-3.5 Turbo Jailbreak Rate: 36. Result ID: 6a0adb6a47f34ce924f14f58. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Method Type":"Black-box Attack Method"}. This record does not assert independent reproduction. ### GCG GPT-3.5 Turbo Jailbreak Rate: 32. Result ID: 6a0adb6a47f34ce924f14f24. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Type":"White-box Attack Method"}. This record does not assert independent reproduction. ### GPTFuzzer GPT-3.5 Turbo Jailbreak Rate: 32. Result ID: 6a0adb6a47f34ce924f14f2b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Type":"Black-box Attack Method"}. This record does not assert independent reproduction. ### GPTFuzzer GPT-3.5 Turbo Jailbreak Rate: 26. Result ID: 6a0adb6a47f34ce924f14f59. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Method Type":"Black-box Attack Method"}. This record does not assert independent reproduction. ### PromptAttack GPT-3.5 Turbo Jailbreak Rate: 24. Result ID: 6a0adb6a47f34ce924f14f5d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Method Type":"Black-box Attack Method"}. This record does not assert independent reproduction. ### COLD-Attack GPT-3.5 Turbo Jailbreak Rate: 22. Result ID: 6a0adb6a47f34ce924f14f27. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Attack Type":"White-box Attack Method"}. This record does not assert independent reproduction. ### MAC GPT-3.5 Turbo Jailbreak Rate: 20. Result ID: 6a0adb6a47f34ce924f14f26. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Type":"White-box Attack Method"}. This record does not assert independent reproduction. ### ArtPrompt GPT-3.5 Turbo Jailbreak Rate: 20. Result ID: 6a0adb6a47f34ce924f14f5c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Method Type":"Black-box Attack Method"}. This record does not assert independent reproduction. ### DRA GPT-3.5 Turbo Jailbreak Rate: 4. Result ID: 6a0adb6a47f34ce924f14f2d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Type":"Black-box Attack Method"}. This record does not assert independent reproduction. ### DRA GPT-3.5 Turbo Jailbreak Rate: 4. Result ID: 6a0adb6a47f34ce924f14f5b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Method Type":"Black-box Attack Method"}. This record does not assert independent reproduction. ### SelfCipher GPT-3.5 Turbo Jailbreak Rate: 2. Result ID: 6a0adb6a47f34ce924f14f30. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Type":"Black-box Attack Method"}. This record does not assert independent reproduction. ### SelfCipher GPT-3.5 Turbo Jailbreak Rate: 0. Result ID: 6a0adb6a47f34ce924f14f5e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Method Type":"Black-box Attack Method"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.