Jailbreak Attack on HarmBench-191 (dev)
97.4Attack Success Rate (ASR)Anchor-based intervention
Evaluation results
| Method | Links | |
|---|---|---|
| Anchor-based interventionModel=Qwen3-4B, Multiplier=2x, Judge=LLM Judge2026.05 | 97.4 | |
| Anchor-based interventionModel=Qwen3-8B, Multiplier=2x, Judge=LlamaGuard2026.05 | 97.4 | |
| Arditi refusal-direction ablationModel=Qwen3-14B, Judge=LLM Judge2026.05 | 97.4 | |
| Constant interventionModel=Qwen3-4B, m*=+18, Judge=LlamaGuard2026.05 | 96.9 | |
| Anchor-based interventionModel=Qwen3-4B, Multiplier=2x, Judge=LlamaGuard2026.05 | 96.9 | |
| Constant interventionModel=Qwen3-8B, m*=+20, Judge=LlamaGuard2026.05 | 96.3 | |
| Anchor-based interventionModel=Qwen3-14B, Multiplier=1x, Judge=LLM Judge2026.05 | 96.3 | |
| Anchor-based interventionModel=Llama-3.1-8B, Multiplier=1x, Judge=LLM Judge2026.05 | 96.3 | |
| Constant interventionModel=Llama-3.1-8B, m*=-4, Judge=LlamaGuard2026.05 | 96.3 | |
| Constant interventionModel=Qwen3-4B, m*=+18, Judge=LLM Judge2026.05 | 95.8 | |
| Constant interventionModel=Qwen3-14B, m*=+40, Judge=LLM Judge2026.05 | 95.8 | |
| Arditi refusal-direction ablationModel=Qwen3-14B, Judge=LlamaGuard2026.05 | 95.8 | |
| Constant interventionModel=Qwen3-14B, m*=+40, Judge=LlamaGuard2026.05 | 95.3 | |
| Constant interventionModel=Llama-3.1-8B, m*=-4, Judge=LLM Judge2026.05 | 95.3 | |
| Anchor-based interventionModel=Llama-3.1-8B, Multiplier=1x, Judge=LlamaGuard2026.05 | 95.3 | |
| Constant interventionModel=Llama-3.1-70B, m*=-8, Judge=LLM Judge2026.05 | 95.3 | |
| Anchor-based interventionModel=Llama-3.1-70B, Multiplier=2x, Judge=LLM Judge2026.05 | 95.3 | |
| Arditi refusal-direction ablationModel=Qwen3-8B, Judge=LLM Judge2026.05 | 94.8 | |
| Anchor-based interventionModel=Qwen3-8B, Multiplier=2x, Judge=LLM Judge2026.05 | 94.2 | |
| Anchor-based interventionModel=Qwen3-14B, Multiplier=1x, Judge=LlamaGuard2026.05 | 94.2 | |
| Arditi refusal-direction ablationModel=Llama-3.1-8B, Judge=LLM Judge2026.05 | 94.2 | |
| Arditi refusal-direction ablationModel=Llama-3.1-8B, Judge=LlamaGuard2026.05 | 94.2 | |
| Arditi refusal-direction ablationModel=Llama-3.1-70B, Judge=LLM Judge2026.05 | 94.2 | |
| Constant interventionModel=Qwen3-8B, m*=+20, Judge=LLM Judge2026.05 | 93.2 | |
| Arditi refusal-direction ablationModel=Qwen3-8B, Judge=LlamaGuard2026.05 | 92.7 | |
| Arditi refusal-direction ablationModel=Qwen3-32B, Judge=LLM Judge2026.05 | 92.1 | |
| Constant interventionModel=Llama-3.1-70B, m*=-8, Judge=LlamaGuard2026.05 | 92.1 | |
| Anchor-based interventionModel=Llama-3.1-70B, Multiplier=2x, Judge=LlamaGuard2026.05 | 92.1 | |
| Arditi refusal-direction ablationModel=Qwen3-4B, Judge=LLM Judge2026.05 | 91.6 | |
| Arditi refusal-direction ablationModel=Qwen3-32B, Judge=LlamaGuard2026.05 | 91.6 | |
| Arditi refusal-direction ablationModel=Qwen3-1.7B, Judge=LlamaGuard2026.05 | 91.1 | |
| Arditi refusal-direction ablationModel=Qwen3-4B, Judge=LlamaGuard2026.05 | 91.1 | |
| Arditi refusal-direction ablationModel=Llama-3.1-70B, Judge=LlamaGuard2026.05 | 91.1 | |
| Constant interventionModel=Qwen3-32B, m*=-80, Judge=LLM Judge2026.05 | 90.6 | |
| Anchor-based interventionModel=Qwen3-32B, Multiplier=2x, Judge=LLM Judge2026.05 | 90.1 | |
| Constant interventionModel=Qwen3-32B, m*=-80, Judge=LlamaGuard2026.05 | 90.1 | |
| Anchor-based interventionModel=Qwen3-32B, Multiplier=2x, Judge=LlamaGuard2026.05 | 89.5 | |
| Constant interventionModel=Qwen3-1.7B, m*=+30, Judge=LlamaGuard2026.05 | 88.5 | |
| Anchor-based interventionModel=Qwen3-1.7B, Multiplier=2x, Judge=LlamaGuard2026.05 | 87.4 | |
| Arditi refusal-direction ablationModel=Qwen3-1.7B, Judge=LLM Judge2026.05 | 85.3 | |
| Anchor-based interventionModel=Qwen3-1.7B, Multiplier=2x, Judge=LLM Judge2026.05 | 78 | |
| Constant interventionModel=Qwen3-1.7B, m*=+30, Judge=LLM Judge2026.05 | 77 |