Agent Safety Judgment on ROME Implicit Risks 1.0 (IR)
86.81F1-ScoreARISE (Std)
Evaluation Results
| Method | Links | |
|---|---|---|
| ARISE (Std)Model=Claude 3.7 Sonnet2026.05 | 86.81 | |
| ARISE (Std)Model=DeepSeek-R12026.05 | 82.47 | |
| Abl. UnsafeModel=Claude 3.7 Sonnet2026.05 | 80.2 | |
| ARISE (Std)Model=DeepSeek-V32026.05 | 76.09 | |
| Abl. UnsafeModel=DeepSeek-R12026.05 | 72.88 | |
| Abl. UnsafeModel=DeepSeek-V32026.05 | 69.23 | |
| ARISE (Std)Model=GPT-4o2026.05 | 67.88 | |
| ARISE (Std)Model=Qwen3 (235B)2026.05 | 66.67 | |
| Abl. FlippedModel=Claude 3.7 Sonnet2026.05 | 66 | |
| Abl. UnsafeModel=GPT-4o2026.05 | 65.2 | |
| Zero-shot (Base)Model=Claude 3.7 Sonnet2026.05 | 63.47 | |
| Abl. UnsafeModel=Qwen3 (235B)2026.05 | 61.79 | |
| Zero-shot (Base)Model=DeepSeek-R12026.05 | 57.72 | |
| ARISE (Std)Model=Qwen3 (8B)2026.05 | 55.71 | |
| Abl. FlippedModel=DeepSeek-R12026.05 | 52.38 | |
| Abl. FlippedModel=DeepSeek-V32026.05 | 50.44 | |
| Abl. UnsafeModel=Qwen3 (8B)2026.05 | 49.4 | |
| Zero-shot (Base)Model=Qwen3 (8B)2026.05 | 42.86 | |
| Zero-shot (Base)Model=Qwen3 (235B)2026.05 | 42.53 | |
| Abl. FlippedModel=GPT-4o2026.05 | 41.63 | |
| Abl. FlippedModel=Qwen3 (235B)2026.05 | 36.19 | |
| Zero-shot (Base)Model=GPT-4o2026.05 | 31.46 | |
| Zero-shot (Base)Model=DeepSeek-V32026.05 | 27.03 | |
| Abl. FlippedModel=Qwen3 (8B)2026.05 | 24.1 |