HarmBench
Benchmarks
TaskDatasetSOTA resultTrendResults
HarmBench target: Llama-3.1-8B
75Attack Success Rate (ASR)
11
HarmBench vani harm
1ASR
10
HarmBench adv harm
6.4ASR
10
HarmBench
0.5AE (Risk/TFLOPs) GCG
10
HarmBench
17DirReq
10
HarmBench Clean
99Clean Rate
10
HarmBench
98.2Score
10
HarmBench
6Inference Time
10
HarmBench
55.92Routing F1
10
HarmBench standard behaviors
61ASR
9
HarmBench (first 200 harmful queries)
92.5ASR (%) (LLaMA-3-8B)
9
HarmBench Standard category
100ASRr
9
HarmBench Open-Source Models
81.5Llama-3.1-8B Success Rate
9
HarmBench Closed-Source Models
86GPT-4o Jailbreak Success Rate
9
HarmBench
100ASR (X-Teaming)
8
HarmBench DirectRequest
57.5Trigger Coverage @8
8
HarmBench target: LLaMA-2-7B
72.81Attack Success Rate (ASR)
8
HarmBench Standard Behaviors Qwen 14B
87.5ASR
8
HarmBench Standard Behaviors Qwen 7B
73.3ASR
8
HarmBench Standard Behaviors Vicuna 13B
59ASR
8
HarmBench Standard Behaviors Vicuna 7B
90ASR
8
HarmBench Standard Behaviors Llama 2 13B
80.5ASR
8
HarmBench Standard Behaviors Llama 2 7B
85ASR
8
HarmBench first 50 behaviors (fixed subset)
0ASR
8
HarmBench
0Additional Tokens
8