List your company
ResearchFormerly WizWand

Agent Safety Judgment on ROME Original 1.0 (Seed)

94.53F1-ScoreAbl. Unsafe
43.8356.992570.15583.3175May 5, 2026
Updated 6mo ago

Evaluation results

MethodLinks
Abl. UnsafeModel=DeepSeek-R12026.0594.53
Abl. UnsafeModel=GPT-4o2026.0593.68
ARISE (Std)Model=DeepSeek-R12026.0592.08
Abl. UnsafeModel=Claude 3.7 Sonnet2026.0590.59
Abl. UnsafeModel=Qwen3 (235B)2026.0589.42
Abl. UnsafeModel=DeepSeek-V32026.0588.24
Abl. UnsafeModel=Qwen3 (8B)2026.0581.82
ARISE (Std)Model=DeepSeek-V32026.0581.48
ARISE (Std)Model=Claude 3.7 Sonnet2026.0580.65
Zero-shot (Base)Model=DeepSeek-R12026.0580.22
Abl. FlippedModel=DeepSeek-R12026.0578.21
Zero-shot (Base)Model=Claude 3.7 Sonnet2026.0577.51
Abl. FlippedModel=Claude 3.7 Sonnet2026.0576.19
ARISE (Std)Model=Qwen3 (8B)2026.0574.61
ARISE (Std)Model=Qwen3 (235B)2026.0574.07
Abl. FlippedModel=DeepSeek-V32026.0573.27
Abl. FlippedModel=Qwen3 (8B)2026.0571.66
Abl. FlippedModel=Qwen3 (235B)2026.0566.01
Abl. FlippedModel=GPT-4o2026.0562.04
ARISE (Std)Model=GPT-4o2026.0560.14
Zero-shot (Base)Model=Qwen3 (8B)2026.0555.09
Zero-shot (Base)Model=Qwen3 (235B)2026.0551.89
Zero-shot (Base)Model=GPT-4o2026.0549.1
Zero-shot (Base)Model=DeepSeek-V32026.0545.78