SafeBench
Benchmarks
TaskDatasetSOTA resultTrendResults
SafeBench
0ASR
245
SafeBench Tiny
100ASR
94
SafeBench
1.7HF
54
SafeBench 100 sampled harmful queries
97ASR
48
Safebench (test)
92IA ASR
20
SafeBench
99Overall Safety Score
19
SafeBench Scenario 6 (Unprotected Left-turn)
0Collision Rate (CR)
16
SafeBench
100ADU Success Rate
16
SafeBench CARLA v3 (scenario-level)
50.7Attack Success Rate
8
SafeBench
30Collision Rate: Straight Obstacle
8
SafeBench
22Collision Rate (CN)
6
SafeBench FigStep (ID)
92.3ASR
6
SafeBench QR (ID)
0ASR
6
SafeBench Mirror (ID)
100ASR
6
SafeBench critical scenarios
2.1Collision Rate (SO)
5
SafeBench
3.26FS ASR
4
SafeBench held-out (test)
5CR (Straight Obstacle)
3
SafeBench evaluated on OpenAI-o1
34.8FS
1