HarmBench
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
HarmBench (val)
82.9Agreement Rate
4
HarmBench (test)
73.92Mean Score
4
HarmBench behaviors
100Attack Success Rate (ASR)
4
HB-A (HarmBench Audio)
99.3Safety Score
4
Harmbench 39 standard behavior examples
0Attack Success Rate
4
Harmbench
23.75ASR
4
HarmBench
6.25Attack Success Rate
3
HarmBench 2024 (test)
65.6Attack Success Probability (ASP)
3
HarmBench Gemma-3-4B-IT (held-out split)
88.6Edit Refusal Rate
3
HarmBench Misinfo. & Disinfo.
80ASR
3
HarmBench General Harm
0Attack Success Rate (ASR)
3
HarmBench V2 (val)
72Attack Success Rate
3
HarmBench 159 standard behaviors (test)
0Attack Success Rate (ASR)
3
HarmBench (test)
26.5MS_R
3
HarmBench (test)
85.1ASR@1
3
HarmBench Claude-4-Sonnet (test)
71ASR@1
3
HarmBench gpt-4o-2024-08-06 (test)
86ASR
3
HarmBench gpt-3.5-turbo-0125 (test)
80ASR
3
HarmBench
95.6F1 Score
2
HarmBench cyber
92.8F1 Score
2
HarmBench (60 prompts)
5,976Total Attempts
2
HarmBench 200 standard behaviors
34.5GCG Success Rate
2
HarmBench
9.7Median ASR
2
HarmBench Qwen3-8B, Reasoning-enabled mode
90ASR
2
HarmBench Qwen3-8B, Non-reasoning mode
100Attack Success Rate (ASR)
2