Multi-turn Safety Risk Assessment on Terminal tasks
96ASRw/o Defense
Evaluation results
| Method | Links | ||
|---|---|---|---|
| w/o DefenseModel=Gemini-1.5-Flash2026.02 | 96 | 0 | |
| FirewallModel=Gemini-1.5-Flash2026.02 | 92 | 4 | |
| w/o DefenseModel=Claude-3.5-Sonnet2026.02 | 76 | 24 | |
| FirewallModel=Claude-3.5-Sonnet2026.02 | 72 | 28 | |
| BaselineModel=Gemini-1.5-Flash2026.02 | 52 | 32 | |
| ToolShieldModel=Gemini-1.5-Flash2026.02 | 44 | 44 | |
| BaselineModel=Claude-3.5-Sonnet2026.02 | 40 | 52 | |
| ToolShieldModel=Claude-3.5-Sonnet2026.02 | 16 | 84 |