List your company
ResearchFormerly WizWand

Agent Safety Judgment on ROME Implicit Risks 1.0 (IR)

86.81F1-ScoreARISE (Std)
21.591638.523355.45572.3867May 5, 2026
Updated 6mo ago

Evaluation results

MethodLinks
ARISE (Std)Model=Claude 3.7 Sonnet2026.0586.81
ARISE (Std)Model=DeepSeek-R12026.0582.47
Abl. UnsafeModel=Claude 3.7 Sonnet2026.0580.2
ARISE (Std)Model=DeepSeek-V32026.0576.09
Abl. UnsafeModel=DeepSeek-R12026.0572.88
Abl. UnsafeModel=DeepSeek-V32026.0569.23
ARISE (Std)Model=GPT-4o2026.0567.88
ARISE (Std)Model=Qwen3 (235B)2026.0566.67
Abl. FlippedModel=Claude 3.7 Sonnet2026.0566
Abl. UnsafeModel=GPT-4o2026.0565.2
Zero-shot (Base)Model=Claude 3.7 Sonnet2026.0563.47
Abl. UnsafeModel=Qwen3 (235B)2026.0561.79
Zero-shot (Base)Model=DeepSeek-R12026.0557.72
ARISE (Std)Model=Qwen3 (8B)2026.0555.71
Abl. FlippedModel=DeepSeek-R12026.0552.38
Abl. FlippedModel=DeepSeek-V32026.0550.44
Abl. UnsafeModel=Qwen3 (8B)2026.0549.4
Zero-shot (Base)Model=Qwen3 (8B)2026.0542.86
Zero-shot (Base)Model=Qwen3 (235B)2026.0542.53
Abl. FlippedModel=GPT-4o2026.0541.63
Abl. FlippedModel=Qwen3 (235B)2026.0536.19
Zero-shot (Base)Model=GPT-4o2026.0531.46
Zero-shot (Base)Model=DeepSeek-V32026.0527.03
Abl. FlippedModel=Qwen3 (8B)2026.0524.1