# Attack Success Rate (ASR) on AdvBench (test) Source: [Attack Success Rate (ASR) on AdvBench (test)]() Collection index: [llms.txt]() Data updated: 2026-05-18T09:28:34.861Z Evaluation of the Attack Success Rate (ASR) for various large language models (including GPT-3.5/4/4o, Claude 3.5, LLaMA 3.1, and Mixtral) against adversarial prompts in the AdvBench (test) dataset. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Jailbreak | | Dataset (including any stated split) | AdvBench (test) | | Metric | ASR (GPT-3.5 Turbo) | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 16 of 16 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [ASR (GPT-3.5 Turbo)](): Higher is better; unit: PERCENT. - [ASR (GPT-4 Turbo)](): Higher is better; unit: PERCENT. - [ASR (GPT-4)](): Higher is better; unit: PERCENT. - [ASR (GPT-4o)](): Higher is better; unit: PERCENT. - [ASR (GPT-4o mini)](): Higher is better; unit: PERCENT. - [ASR (Claude 3.5 Sonnet)](): Higher is better; unit: PERCENT. - [ASR (LLaMA 3.1 405B)](): Higher is better; unit: PERCENT. - [ASR (Mixtral 8x22B)](): Higher is better; unit: PERCENT. - [Average ASR](): Higher is better; unit: PERCENT. ## Recorded results | Method | ASR (GPT-3.5 Turbo) | Result date | Report paper | | --- | --- | --- | --- | | ReNeLLM | 91.35 | Not recorded | 68fa751a1e209131cea65ca8 | | FlipAttack | 88.65 | Not recorded | 68fa751a1e209131cea65ca8 | | CodeChameleon | 84.62 | Not recorded | 68fa751a1e209131cea65ca8 | | AutoDAN | 81.73 | Not recorded | 68fa751a1e209131cea65ca8 | | TAP | 60.54 | Not recorded | 68fa751a1e209131cea65ca8 | | PAIR | 59.68 | Not recorded | 68fa751a1e209131cea65ca8 | | Base64 | 45 | Not recorded | 68fa751a1e209131cea65ca8 | | GCG | 42.88 | Not recorded | 68fa751a1e209131cea65ca8 | | DeepInception | 41.13 | Not recorded | 68fa751a1e209131cea65ca8 | | GPTFuzzer | 37.79 | Not recorded | 68fa751a1e209131cea65ca8 | | MAC | 36.15 | Not recorded | 68fa751a1e209131cea65ca8 | | COLD-Attack | 34.23 | Not recorded | 68fa751a1e209131cea65ca8 | | ArtPrompt | 14.06 | Not recorded | 68fa751a1e209131cea65ca8 | | PromptAttack | 13.46 | Not recorded | 68fa751a1e209131cea65ca8 | | DRA | 9.42 | Not recorded | 68fa751a1e209131cea65ca8 | | SelfCipher | 0 | Not recorded | 68fa751a1e209131cea65ca8 | ### ReNeLLM ASR (GPT-3.5 Turbo): 91.35. Result ID: 6a0adb6a47f34ce924f14f22. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Method Category":"Black-box Attack Method"}. This record does not assert independent reproduction. ### FlipAttack ASR (GPT-3.5 Turbo): 88.65. Result ID: 6a0adb6a47f34ce924f14f23. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Method Category":"Black-box Attack Method"}. This record does not assert independent reproduction. ### CodeChameleon ASR (GPT-3.5 Turbo): 84.62. Result ID: 6a0adb6a47f34ce924f14f21. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Method Category":"Black-box Attack Method"}. This record does not assert independent reproduction. ### AutoDAN ASR (GPT-3.5 Turbo): 81.73. Result ID: 6a0adb6a47f34ce924f14f15. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Method Category":"White-box Attack Method"}. This record does not assert independent reproduction. ### TAP ASR (GPT-3.5 Turbo): 60.54. Result ID: 6a0adb6a47f34ce924f14f19. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Method Category":"Black-box Attack Method"}. This record does not assert independent reproduction. ### PAIR ASR (GPT-3.5 Turbo): 59.68. Result ID: 6a0adb6a47f34ce924f14f18. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Method Category":"Black-box Attack Method"}. This record does not assert independent reproduction. ### Base64 ASR (GPT-3.5 Turbo): 45. Result ID: 6a0adb6a47f34ce924f14f1a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Method Category":"Black-box Attack Method"}. This record does not assert independent reproduction. ### GCG ASR (GPT-3.5 Turbo): 42.88. Result ID: 6a0adb6a47f34ce924f14f14. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Method Category":"White-box Attack Method"}. This record does not assert independent reproduction. ### DeepInception ASR (GPT-3.5 Turbo): 41.13. Result ID: 6a0adb6a47f34ce924f14f1c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Method Category":"Black-box Attack Method"}. This record does not assert independent reproduction. ### GPTFuzzer ASR (GPT-3.5 Turbo): 37.79. Result ID: 6a0adb6a47f34ce924f14f1b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Method Category":"Black-box Attack Method"}. This record does not assert independent reproduction. ### MAC ASR (GPT-3.5 Turbo): 36.15. Result ID: 6a0adb6a47f34ce924f14f16. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Method Category":"White-box Attack Method"}. This record does not assert independent reproduction. ### COLD-Attack ASR (GPT-3.5 Turbo): 34.23. Result ID: 6a0adb6a47f34ce924f14f17. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Attack Method Category":"White-box Attack Method"}. This record does not assert independent reproduction. ### ArtPrompt ASR (GPT-3.5 Turbo): 14.06. Result ID: 6a0adb6a47f34ce924f14f1e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Method Category":"Black-box Attack Method"}. This record does not assert independent reproduction. ### PromptAttack ASR (GPT-3.5 Turbo): 13.46. Result ID: 6a0adb6a47f34ce924f14f1f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Method Category":"Black-box Attack Method"}. This record does not assert independent reproduction. ### DRA ASR (GPT-3.5 Turbo): 9.42. Result ID: 6a0adb6a47f34ce924f14f1d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Method Category":"Black-box Attack Method"}. This record does not assert independent reproduction. ### SelfCipher ASR (GPT-3.5 Turbo): 0. Result ID: 6a0adb6a47f34ce924f14f20. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Attack Method Category":"Black-box Attack Method"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.