AdvBench
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
AdvBench GPT-4o 1.0 (test)
92.67ASR
17
AdvBench GPT-4-turbo 1.0 (test)
9,846ASR
17
AdvBench GPT-4 1.0 (test)
0.77ASR
17
AdvBench N=520
0.2Flip Rate
16
AdvBench Vicuna-7B
95.7AUROC
16
AdvBench Mistral-7B
99AUROC
16
AdvBench Llama3-8B
0.967AUROC
16
AdvBench
100Attack Success Rate (ASR)
16
AdvBench
0.16Loss
16
AdvBench (test)
91.35ASR (GPT-3.5 Turbo)
16
AdvBench
88.27ASR-M
16
AdvBench-M OOD
61.3ASR (OOD)
16
AdvBench (test)
1Harmful Score
16
AdvBench-audio Harmful
26.35ASR Score
15
AdvBench 50 harmful queries
100Success Rate
15
AdvBench
0Thinking Collapse Rate
15
AdvBench (test)
73.86GCG Success Rate
15
AdvBench Ensemble configuration Average
1.43Harmfulness Score (HS)
15
AdvBench Ensemble configuration Llama-3-70B
4.6Harmfulness Score (HS)
15
AdvBench Ensemble configuration Claude-v2
1.08Harmfulness Score (HS)
15
AdvBench AutoDAN attack
100DSR
15
AdvBench GCG attack
100DSR
15
AdvBench
99.9ASP
14
AdvBench
0.2JSR (Jailbreak Success Rate)
14
AdvBench 50 harmful behaviors
100ASR
14