JBB-Behaviors
Benchmarks
TaskDatasetSOTA resultTrendResults
JBB-Behaviors
0ASR
121
JBB-Behaviors utilitarian dilemmas (test)
87Jailbreak Success Rate
72
JBB-Behaviors
100Rule-Judge Score
56
JBB Behaviors
100ASR
35
JBB Behaviors
100ASR
35
JBB-Behaviors (test)
99ASR (GPT-4o)
27
JBB-Behaviors 100 harmful prompts (test)
100Refusal Rate
24
JBB-Behaviors (test)
0ASR
24
JBB-Behaviors Scenario J3
0.707Hypervolume
21
JBB-Behaviors Scenario J2
0.691Hypervolume
21
JBB-Behaviors Scenario J1
59.1Hypervolume
21
JBB-Behaviors (test)
0ASR (Guardrail Model)
20
JBB-Behaviors
10ASR (PAIR)
18
JBB-Behaviors
0.3ASR (PAIR, Guardrail Model)
18
JBB-Behaviors
99.2ASR (GPT-4o)
12
JBB-Behaviors GPT-4o
93Attack Success Rate (ASR)
11
JBB-Behaviors GPT-3.5-Turbo
98ASR
11
JBB-Behaviors DeepSeek-R1-14B
98ASR
11
JBB-Behaviors Qwen2.5-14B
98ASR
11
JBB-Behaviors Llama-3.3-70B
99Attack Success Rate (ASR)
11
JBB-Behaviors
99.3Safety Score
9
JBB-Behaviors n = 200 (blind)
0.828F1 Score
8
JBB-Behaviors (held-out)
93.7AUROC
5
JBB-Behaviors
0Unsafe Interaction Rate
3
JBB-Behaviors Benign
99.96Accuracy
1