Agent Safety Judgment on ROME Original 1.0 (Seed)
94.53F1-ScoreAbl. Unsafe
Evaluation Results
| Method | Links | |
|---|---|---|
| Abl. UnsafeModel=DeepSeek-R12026.05 | 94.53 | |
| Abl. UnsafeModel=GPT-4o2026.05 | 93.68 | |
| ARISE (Std)Model=DeepSeek-R12026.05 | 92.08 | |
| Abl. UnsafeModel=Claude 3.7 Sonnet2026.05 | 90.59 | |
| Abl. UnsafeModel=Qwen3 (235B)2026.05 | 89.42 | |
| Abl. UnsafeModel=DeepSeek-V32026.05 | 88.24 | |
| Abl. UnsafeModel=Qwen3 (8B)2026.05 | 81.82 | |
| ARISE (Std)Model=DeepSeek-V32026.05 | 81.48 | |
| ARISE (Std)Model=Claude 3.7 Sonnet2026.05 | 80.65 | |
| Zero-shot (Base)Model=DeepSeek-R12026.05 | 80.22 | |
| Abl. FlippedModel=DeepSeek-R12026.05 | 78.21 | |
| Zero-shot (Base)Model=Claude 3.7 Sonnet2026.05 | 77.51 | |
| Abl. FlippedModel=Claude 3.7 Sonnet2026.05 | 76.19 | |
| ARISE (Std)Model=Qwen3 (8B)2026.05 | 74.61 | |
| ARISE (Std)Model=Qwen3 (235B)2026.05 | 74.07 | |
| Abl. FlippedModel=DeepSeek-V32026.05 | 73.27 | |
| Abl. FlippedModel=Qwen3 (8B)2026.05 | 71.66 | |
| Abl. FlippedModel=Qwen3 (235B)2026.05 | 66.01 | |
| Abl. FlippedModel=GPT-4o2026.05 | 62.04 | |
| ARISE (Std)Model=GPT-4o2026.05 | 60.14 | |
| Zero-shot (Base)Model=Qwen3 (8B)2026.05 | 55.09 | |
| Zero-shot (Base)Model=Qwen3 (235B)2026.05 | 51.89 | |
| Zero-shot (Base)Model=GPT-4o2026.05 | 49.1 | |
| Zero-shot (Base)Model=DeepSeek-V32026.05 | 45.78 |