Agent Safety Judgment on ROME Shortcut Decision-Making 1.0
94.53F1-ScoreARISE (Std)
Evaluation Results
| Method | Links | |
|---|---|---|
| ARISE (Std)Model=Claude 3.7 Sonnet2026.05 | 94.53 | |
| ARISE (Std)Model=DeepSeek-R12026.05 | 88.67 | |
| ARISE (Std)Model=GPT-4o2026.05 | 88.21 | |
| ARISE (Std)Model=DeepSeek-V32026.05 | 87.56 | |
| Abl. FlippedModel=Claude 3.7 Sonnet2026.05 | 86.36 | |
| Abl. UnsafeModel=Claude 3.7 Sonnet2026.05 | 86.36 | |
| Zero-shot (Base)Model=Claude 3.7 Sonnet2026.05 | 85.97 | |
| Abl. FlippedModel=DeepSeek-V32026.05 | 84.55 | |
| Abl. FlippedModel=DeepSeek-R12026.05 | 81.18 | |
| Abl. UnsafeModel=DeepSeek-R12026.05 | 77.87 | |
| Abl. UnsafeModel=DeepSeek-V32026.05 | 77.6 | |
| Abl. UnsafeModel=GPT-4o2026.05 | 74.9 | |
| Abl. FlippedModel=GPT-4o2026.05 | 71.24 | |
| ARISE (Std)Model=Qwen3 (235B)2026.05 | 70.65 | |
| Zero-shot (Base)Model=DeepSeek-V32026.05 | 63.83 | |
| Zero-shot (Base)Model=DeepSeek-R12026.05 | 62.75 | |
| Abl. UnsafeModel=Qwen3 (235B)2026.05 | 61.73 | |
| ARISE (Std)Model=Qwen3 (8B)2026.05 | 61.22 | |
| Abl. UnsafeModel=Qwen3 (8B)2026.05 | 54.65 | |
| Abl. FlippedModel=Qwen3 (235B)2026.05 | 51.37 | |
| Zero-shot (Base)Model=GPT-4o2026.05 | 47.96 | |
| Zero-shot (Base)Model=Qwen3 (8B)2026.05 | 46.84 | |
| Zero-shot (Base)Model=Qwen3 (235B)2026.05 | 43.43 | |
| Abl. FlippedModel=Qwen3 (8B)2026.05 | 40.79 |