AdvBench
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
AdvBench Gemini-2.5-Flash
71.3ASR
7
AdvBench target: o4-mini
64ASR (o4-mini)
7
AdvBench
18ASR (GPT-4o mini)
7
AdvBench Llama-3-8B
44ASR
7
AdvBench Claude-3.5-Sonnet
86ASR
7
AdvBench GPT-4o
6ASR
7
AdvBench
99.4Refusal Rate (String Match)
6
AdvBench
95.8Ref Rate
6
AdvBench 100-instance sample
2.6Average Queries
6
AdvBench 14 (test)
97.02ASR
6
AdvBench o1-preview
0.01Attack Success Rate (ASR)
6
AdvBench GPT-oss-20B target (test)
85.3ASR
6
AdvBench Qwen 3-14B target (test)
51.92ASR
6
AdvBench Gemma 3-12B target (test)
100ASR
6
AdvBench Llama 3.2-11B target (test)
95.51ASR
6
AdvBench Llama-3-Multi-layer defense
1.34ASR
5
AdvBench Llama-3-Uncensored
99.62ASR
5
AdvBench Average across Victims
88.2Attack Success Rate (ASR)
5
AdvBench LLaMA-3.1-70B
39.7ASR (SMO, GPT-4o)
5
AdvBench Phi-3 Medium 14B Instruct
41ASR (SMO, GPT-4o)
5
AdvBench LLaMA-2-7B-Chat
40ASR (SMO, GPT-4o)
5
AdvBench GPT-4 Series
98.9ASR
5
AdvBench Llama2-13b
95.4ASR
5
AdvBench Llama2-7b
97.3Attack Success Rate (ASR)
5
AdvBench
35ASR-G
5