List your company
ResearchFormerly WizWand

Agent Safety Judgment on ROME Contextual Ambiguity 1.0

91.92F1 ScoreARISE (Std)
42.665655.452868.2481.0272May 5, 2026
Updated 6mo ago

Evaluation results

MethodLinks
ARISE (Std)Model=Claude 3.7 Sonnet2026.0591.92
ARISE (Std)Model=DeepSeek-R12026.0587
ARISE (Std)Model=GPT-4o2026.0585.26
Abl. UnsafeModel=Claude 3.7 Sonnet2026.0584.16
Abl. FlippedModel=DeepSeek-R12026.0583.13
ARISE (Std)Model=DeepSeek-V32026.0582.13
Zero-shot (Base)Model=DeepSeek-R12026.0580.22
Abl. FlippedModel=Claude 3.7 Sonnet2026.0579.6
ARISE (Std)Model=Qwen3 (235B)2026.0578.61
Abl. UnsafeModel=DeepSeek-R12026.0577.73
Zero-shot (Base)Model=Claude 3.7 Sonnet2026.0574.47
Abl. FlippedModel=DeepSeek-V32026.0573.68
Abl. UnsafeModel=GPT-4o2026.0573.03
Abl. UnsafeModel=DeepSeek-V32026.0572.5
ARISE (Std)Model=Qwen3 (8B)2026.0571.7
Abl. UnsafeModel=Qwen3 (8B)2026.0568.13
Abl. UnsafeModel=Qwen3 (235B)2026.0567.72
Abl. FlippedModel=GPT-4o2026.0561.75
Abl. FlippedModel=Qwen3 (235B)2026.0557.89
Zero-shot (Base)Model=Qwen3 (8B)2026.0557.14
Zero-shot (Base)Model=Qwen3 (235B)2026.0557.14
Abl. FlippedModel=Qwen3 (8B)2026.0548.75
Zero-shot (Base)Model=DeepSeek-V32026.0547.62
Zero-shot (Base)Model=GPT-4o2026.0544.56