# Jailbreaking Attack on AdvBench (ASR + GPT) Source: [Jailbreaking Attack on AdvBench (ASR + GPT)]() Collection index: [llms.txt]() Data updated: 2026-03-04T17:14:10.745Z Evaluation of model susceptibility to jailbreak attacks using the AdvBench dataset, specifically reporting Attack Success Rate (ASR) and ASR measured using GPT-based evaluation. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Jailbreaking Attack | | Dataset (including any stated split) | AdvBench | | Metric | ASR | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 28 of 28 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [ASR](): Higher is better; unit: PERCENT. - [ASR (GPT)](): Higher is better; unit: PERCENT. ## Recorded results | Method | ASR | Result date | Report paper | | --- | --- | --- | --- | | PiF | 97.7 | Not recorded | 694cc4e953fdd5c36173a68a | | CHaRS | 92.31 | Not recorded | 69a7bc6a2d603e6de3e3d308 | | CHaRS-PCT | 92.31 | Not recorded | 69a7bc6a2d603e6de3e3d308 | | CHaRS | 92.31 | Not recorded | 69a7bc6a2d603e6de3e3d308 | | CHaRS-PCT | 92.31 | Not recorded | 69a7bc6a2d603e6de3e3d308 | | APAs | 92 | Not recorded | 694cc4e953fdd5c36173a68a | | DirAbl | 91.35 | Not recorded | 69a7bc6a2d603e6de3e3d308 | | CHaRS-PCT | 91.35 | Not recorded | 69a7bc6a2d603e6de3e3d308 | | DirAbl | 90.38 | Not recorded | 69a7bc6a2d603e6de3e3d308 | | CHaRS | 90.38 | Not recorded | 69a7bc6a2d603e6de3e3d308 | | DirAbl | 89.42 | Not recorded | 69a7bc6a2d603e6de3e3d308 | | CHaRS | 86.54 | Not recorded | 69a7bc6a2d603e6de3e3d308 | | CHaRS-PCT | 86.54 | Not recorded | 69a7bc6a2d603e6de3e3d308 | | RIPPLE | 86 | Not recorded | 694cc4e953fdd5c36173a68a | | CHaRS-PCT | 85.58 | Not recorded | 69a7bc6a2d603e6de3e3d308 | | CHaRS | 85.58 | Not recorded | 69a7bc6a2d603e6de3e3d308 | | CHaRS | 84.62 | Not recorded | 69a7bc6a2d603e6de3e3d308 | | DirAbl | 83.65 | Not recorded | 69a7bc6a2d603e6de3e3d308 | | CHaRS-PCT | 83.65 | Not recorded | 69a7bc6a2d603e6de3e3d308 | | DirAbl | 82.69 | Not recorded | 69a7bc6a2d603e6de3e3d308 | | CHaRS | 80.77 | Not recorded | 69a7bc6a2d603e6de3e3d308 | | CHaRS-PCT | 80.77 | Not recorded | 69a7bc6a2d603e6de3e3d308 | | DirAbl | 79.81 | Not recorded | 69a7bc6a2d603e6de3e3d308 | | DirAbl | 78.85 | Not recorded | 69a7bc6a2d603e6de3e3d308 | | ReNeLLM | 71.6 | Not recorded | 694cc4e953fdd5c36173a68a | | LLM-FUZZER | 60 | Not recorded | 694cc4e953fdd5c36173a68a | | AutoDAN | 26.4 | Not recorded | 694cc4e953fdd5c36173a68a | ### PiF ASR: 97.7. Result ID: 699941505698665ec4ab6216. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Target Model":"GPT-4"}. This record does not assert independent reproduction. ### CHaRS ASR: 92.31. Result ID: 69a868526dbb7075b9502e7c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Llama3.1-8B-Instruct"}. This record does not assert independent reproduction. ### CHaRS-PCT ASR: 92.31. Result ID: 69a868526dbb7075b9502e7d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Llama3.1-8B-Instruct"}. This record does not assert independent reproduction. ### CHaRS ASR: 92.31. Result ID: 69a868526dbb7075b9502e82. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-3B-Instruct"}. This record does not assert independent reproduction. ### CHaRS-PCT ASR: 92.31. Result ID: 69a868526dbb7075b9502e83. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-3B-Instruct"}. This record does not assert independent reproduction. ### APAs ASR: 92. Result ID: 699941505698665ec4ab6210. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Target Model":"GPT-4"}. This record does not assert independent reproduction. ### DirAbl ASR: 91.35. Result ID: 69a868526dbb7075b9502e7b. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Base Model":"Llama3.1-8B-Instruct"}. This record does not assert independent reproduction. ### CHaRS-PCT ASR: 91.35. Result ID: 69a868526dbb7075b9502e86. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### DirAbl ASR: 90.38. Result ID: 69a868526dbb7075b9502e81. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-3B-Instruct"}. This record does not assert independent reproduction. ### CHaRS ASR: 90.38. Result ID: 69a868526dbb7075b9502e85. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### DirAbl ASR: 89.42. Result ID: 69a868526dbb7075b9502e84. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B-Instruct"}. This record does not assert independent reproduction. ### CHaRS ASR: 86.54. Result ID: 69a868526dbb7075b9502e7f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Llama3.2-3B-Instruct"}. This record does not assert independent reproduction. ### CHaRS-PCT ASR: 86.54. Result ID: 69a868526dbb7075b9502e80. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Llama3.2-3B-Instruct"}. This record does not assert independent reproduction. ### RIPPLE ASR: 86. Result ID: 699941505698665ec4ab6212. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Target Model":"GPT-4"}. This record does not assert independent reproduction. ### CHaRS-PCT ASR: 85.58. Result ID: 69a868526dbb7075b9502e7a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Gemma2-9B-Instruct"}. This record does not assert independent reproduction. ### CHaRS ASR: 85.58. Result ID: 69a868526dbb7075b9502e8b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-32B-Instruct"}. This record does not assert independent reproduction. ### CHaRS ASR: 84.62. Result ID: 69a868526dbb7075b9502e79. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Gemma2-9B-Instruct"}. This record does not assert independent reproduction. ### DirAbl ASR: 83.65. Result ID: 69a868526dbb7075b9502e7e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Base Model":"Llama3.2-3B-Instruct"}. This record does not assert independent reproduction. ### CHaRS-PCT ASR: 83.65. Result ID: 69a868526dbb7075b9502e8c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-32B-Instruct"}. This record does not assert independent reproduction. ### DirAbl ASR: 82.69. Result ID: 69a868526dbb7075b9502e8a. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-32B-Instruct"}. This record does not assert independent reproduction. ### CHaRS ASR: 80.77. Result ID: 69a868526dbb7075b9502e88. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-14B-Instruct"}. This record does not assert independent reproduction. ### CHaRS-PCT ASR: 80.77. Result ID: 69a868526dbb7075b9502e89. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-14B-Instruct"}. This record does not assert independent reproduction. ### DirAbl ASR: 79.81. Result ID: 69a868526dbb7075b9502e78. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Base Model":"Gemma2-9B-Instruct"}. This record does not assert independent reproduction. ### DirAbl ASR: 78.85. Result ID: 69a868526dbb7075b9502e87. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-14B-Instruct"}. This record does not assert independent reproduction. ### ReNeLLM ASR: 71.6. Result ID: 699941505698665ec4ab6215. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Target Model":"GPT-4"}. This record does not assert independent reproduction. ### LLM-FUZZER ASR: 60. Result ID: 699941505698665ec4ab6213. [Reporting paper]() Recorded evaluation configuration: {"Target Model":"GPT-4"}. This record does not assert independent reproduction. ### AutoDAN ASR: 26.4. Result ID: 699941505698665ec4ab6214. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Target Model":"GPT-4"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.