HarmBench
Benchmarks
TaskDatasetSOTA resultTrendResults
HarmBench Classifier (test)
88.6TASR@1
37
HarmBench (HB) (standard split)
90.57ASR@1
33
Harmbench
0.3ASR
32
HarmBench (test)
0ASR
31
HarmBench
94.85F1 Score
28
HarmBench
5.6ASR
28
HarmBench (test)
97ASR (GPT-4o)
27
HarmBench (test)
31.18ZS
27
HarmBench
1.12Harmful Response Ratio
27
HarmBench out-of-domain
0.62ASR (OM)
26
Harmbench
87.2F1 Score
26
HarmBench Contextual Trajectory Evaluation Multi-turn
96.1SFR
24
HarmBench (test)
100ASR@1000
24
HarmBench (test)
100F1 Score
22
Harmbench (test)
100MCA Accuracy
21
HarmBench
0.889AUROC
20
HarmBench 51 (test)
98.1ASR@5 (Standard)
19
HarmBench n = 200
16ASR
18
HarmBench 1.0 (test)
3.13GCG Attack Success Rate
18
HarmBench (HB)
98.4Evaluation Score (avg@4)
18
HarmBench
95MD Score
18
HarmBench (400 random samples)
0ASR
18
HarmBench example-based Llama3 8B
5Attack Success Rate
17
HarmBench text (test N = 320)
93.75ASR-M
16
HarmBench translated (test)
11Success Rate (EN)
16