List your company
ResearchFormerly WizWand

Agent Safety Judgment on ROME Shortcut Decision-Making 1.0

94.53F1-ScoreARISE (Std)
38.640453.150267.6682.1698May 5, 2026
Updated 6mo ago

Evaluation results

MethodLinks
ARISE (Std)Model=Claude 3.7 Sonnet2026.0594.53
ARISE (Std)Model=DeepSeek-R12026.0588.67
ARISE (Std)Model=GPT-4o2026.0588.21
ARISE (Std)Model=DeepSeek-V32026.0587.56
Abl. FlippedModel=Claude 3.7 Sonnet2026.0586.36
Abl. UnsafeModel=Claude 3.7 Sonnet2026.0586.36
Zero-shot (Base)Model=Claude 3.7 Sonnet2026.0585.97
Abl. FlippedModel=DeepSeek-V32026.0584.55
Abl. FlippedModel=DeepSeek-R12026.0581.18
Abl. UnsafeModel=DeepSeek-R12026.0577.87
Abl. UnsafeModel=DeepSeek-V32026.0577.6
Abl. UnsafeModel=GPT-4o2026.0574.9
Abl. FlippedModel=GPT-4o2026.0571.24
ARISE (Std)Model=Qwen3 (235B)2026.0570.65
Zero-shot (Base)Model=DeepSeek-V32026.0563.83
Zero-shot (Base)Model=DeepSeek-R12026.0562.75
Abl. UnsafeModel=Qwen3 (235B)2026.0561.73
ARISE (Std)Model=Qwen3 (8B)2026.0561.22
Abl. UnsafeModel=Qwen3 (8B)2026.0554.65
Abl. FlippedModel=Qwen3 (235B)2026.0551.37
Zero-shot (Base)Model=GPT-4o2026.0547.96
Zero-shot (Base)Model=Qwen3 (8B)2026.0546.84
Zero-shot (Base)Model=Qwen3 (235B)2026.0543.43
Abl. FlippedModel=Qwen3 (8B)2026.0540.79