# Refusal mechanism ablation on JBB-Behaviors 100 harmful prompts (test) Source: [Refusal mechanism ablation on JBB-Behaviors 100 harmful prompts (test)]() Collection index: [llms.txt]() Data updated: 2026-05-13T10:56:30.871Z Evaluation of model refusal mechanisms using the specific '100 harmful prompts' test split of the JBB-Behaviors dataset, measured by Refusal rate and response Quality. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Refusal mechanism ablation | | Dataset (including any stated split) | JBB-Behaviors 100 harmful prompts (test) | | Metric | Refusal Rate | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 24 of 24 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Refusal Rate](): Higher is better; unit: PERCENT. - [Quality](): Higher is better; unit: SCALAR. ## Recorded results | Method | Refusal Rate | Result date | Report paper | | --- | --- | --- | --- | | CAA | 100 | Not recorded | 6a0405e0dbc836b87d8278f2 | | CAA | 98 | Not recorded | 6a0405e0dbc836b87d8278f2 | | Baseline | 97 | Not recorded | 6a0405e0dbc836b87d8278f2 | | Baseline | 92.9 | Not recorded | 6a0405e0dbc836b87d8278f2 | | Baseline | 88.9 | Not recorded | 6a0405e0dbc836b87d8278f2 | | Baseline | 82.8 | Not recorded | 6a0405e0dbc836b87d8278f2 | | Baseline | 72.7 | Not recorded | 6a0405e0dbc836b87d8278f2 | | Baseline | 65.7 | Not recorded | 6a0405e0dbc836b87d8278f2 | | Baseline | 57.6 | Not recorded | 6a0405e0dbc836b87d8278f2 | | Baseline | 43.4 | Not recorded | 6a0405e0dbc836b87d8278f2 | | CAA | 38.4 | Not recorded | 6a0405e0dbc836b87d8278f2 | | CNA | 34.3 | Not recorded | 6a0405e0dbc836b87d8278f2 | | CNA | 26.3 | Not recorded | 6a0405e0dbc836b87d8278f2 | | CNA | 26.3 | Not recorded | 6a0405e0dbc836b87d8278f2 | | CNA | 20.2 | Not recorded | 6a0405e0dbc836b87d8278f2 | | CNA | 13.1 | Not recorded | 6a0405e0dbc836b87d8278f2 | | CNA | 12.1 | Not recorded | 6a0405e0dbc836b87d8278f2 | | CNA | 5.1 | Not recorded | 6a0405e0dbc836b87d8278f2 | | CAA | 5.1 | Not recorded | 6a0405e0dbc836b87d8278f2 | | CNA | 5.1 | Not recorded | 6a0405e0dbc836b87d8278f2 | | CAA | 0 | Not recorded | 6a0405e0dbc836b87d8278f2 | | CAA | 0 | Not recorded | 6a0405e0dbc836b87d8278f2 | | CAA | 0 | Not recorded | 6a0405e0dbc836b87d8278f2 | | CAA | 0 | Not recorded | 6a0405e0dbc836b87d8278f2 | ### CAA Refusal Rate: 100. Result ID: 6a0458c447f34ce924eb0b25. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-1.5B-Instruct","Steering Strength (alpha)":"1.0"}. This record does not assert independent reproduction. ### CAA Refusal Rate: 98. Result ID: 6a0458c447f34ce924eb0b2e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-72B-Instruct","Steering Strength (alpha)":"1.0"}. This record does not assert independent reproduction. ### Baseline Refusal Rate: 97. Result ID: 6a0458c447f34ce924eb0b23. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-1.5B-Instruct","Steering Strength (alpha)":"0.0"}. This record does not assert independent reproduction. ### Baseline Refusal Rate: 92.9. Result ID: 6a0458c447f34ce924eb0b26. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-3B-Instruct","Steering Strength (alpha)":"0.0"}. This record does not assert independent reproduction. ### Baseline Refusal Rate: 88.9. Result ID: 6a0458c447f34ce924eb0b1d. [Reporting paper]() Recorded evaluation configuration: {"Model":"Llama-3.1-8B-Instruct","Steering Strength (alpha)":"0.0"}. This record does not assert independent reproduction. ### Baseline Refusal Rate: 82.8. Result ID: 6a0458c447f34ce924eb0b29. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-7B-Instruct","Steering Strength (alpha)":"0.0"}. This record does not assert independent reproduction. ### Baseline Refusal Rate: 72.7. Result ID: 6a0458c447f34ce924eb0b20. [Reporting paper]() Recorded evaluation configuration: {"Model":"Llama-3.1-70B-Instruct","Steering Strength (alpha)":"0.0"}. This record does not assert independent reproduction. ### Baseline Refusal Rate: 65.7. Result ID: 6a0458c447f34ce924eb0b2c. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-72B-Instruct","Steering Strength (alpha)":"0.0"}. This record does not assert independent reproduction. ### Baseline Refusal Rate: 57.6. Result ID: 6a0458c447f34ce924eb0b1a. [Reporting paper]() Recorded evaluation configuration: {"Model":"Llama-3.2-3B-Instruct","Steering Strength (alpha)":"0.0"}. This record does not assert independent reproduction. ### Baseline Refusal Rate: 43.4. Result ID: 6a0458c447f34ce924eb0b17. [Reporting paper]() Recorded evaluation configuration: {"Model":"Llama-3.2-1B-Instruct","Steering Strength (alpha)":"0.0"}. This record does not assert independent reproduction. ### CAA Refusal Rate: 38.4. Result ID: 6a0458c447f34ce924eb0b1f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3.1-8B-Instruct","Steering Strength (alpha)":"1.0"}. This record does not assert independent reproduction. ### CNA Refusal Rate: 34.3. Result ID: 6a0458c447f34ce924eb0b27. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Qwen2.5-3B-Instruct","Steering Strength (alpha)":"1.0"}. This record does not assert independent reproduction. ### CNA Refusal Rate: 26.3. Result ID: 6a0458c447f34ce924eb0b1b. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Llama-3.2-3B-Instruct","Steering Strength (alpha)":"1.0"}. This record does not assert independent reproduction. ### CNA Refusal Rate: 26.3. Result ID: 6a0458c447f34ce924eb0b24. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Qwen2.5-1.5B-Instruct","Steering Strength (alpha)":"1.0"}. This record does not assert independent reproduction. ### CNA Refusal Rate: 20.2. Result ID: 6a0458c447f34ce924eb0b18. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Llama-3.2-1B-Instruct","Steering Strength (alpha)":"1.0"}. This record does not assert independent reproduction. ### CNA Refusal Rate: 13.1. Result ID: 6a0458c447f34ce924eb0b2a. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Qwen2.5-7B-Instruct","Steering Strength (alpha)":"1.0"}. This record does not assert independent reproduction. ### CNA Refusal Rate: 12.1. Result ID: 6a0458c447f34ce924eb0b21. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Llama-3.1-70B-Instruct","Steering Strength (alpha)":"1.0"}. This record does not assert independent reproduction. ### CNA Refusal Rate: 5.1. Result ID: 6a0458c447f34ce924eb0b1e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Llama-3.1-8B-Instruct","Steering Strength (alpha)":"1.0"}. This record does not assert independent reproduction. ### CAA Refusal Rate: 5.1. Result ID: 6a0458c447f34ce924eb0b2b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-7B-Instruct","Steering Strength (alpha)":"1.0"}. This record does not assert independent reproduction. ### CNA Refusal Rate: 5.1. Result ID: 6a0458c447f34ce924eb0b2d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Qwen2.5-72B-Instruct","Steering Strength (alpha)":"1.0"}. This record does not assert independent reproduction. ### CAA Refusal Rate: 0. Result ID: 6a0458c447f34ce924eb0b19. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3.2-1B-Instruct","Steering Strength (alpha)":"1.0"}. This record does not assert independent reproduction. ### CAA Refusal Rate: 0. Result ID: 6a0458c447f34ce924eb0b1c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3.2-3B-Instruct","Steering Strength (alpha)":"1.0"}. This record does not assert independent reproduction. ### CAA Refusal Rate: 0. Result ID: 6a0458c447f34ce924eb0b22. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3.1-70B-Instruct","Steering Strength (alpha)":"1.0"}. This record does not assert independent reproduction. ### CAA Refusal Rate: 0. Result ID: 6a0458c447f34ce924eb0b28. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-3B-Instruct","Steering Strength (alpha)":"1.0"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.