Multi-turn Safety Risk Assessment on Multi-turn Safety 100 random sampled tasks
0.15ASRToolShield
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ToolShieldModel=Claude-3.5-Sonnet2026.02 | 0.15 | 0.83 | |
| BaselineModel=Claude-3.5-Sonnet2026.02 | 0.42 | 0.5 | |
| ToolShieldModel=Gemini-1.5-Flash2026.02 | 0.57 | 0.31 | |
| BaselineModel=Gemini-1.5-Flash2026.02 | 0.69 | 0.16 | |
| FirewallModel=Gemini-1.5-Flash2026.02 | 0.79 | 0.11 | |
| w/o DefenseModel=Gemini-1.5-Flash2026.02 | 0.81 | 0.08 | |
| FirewallModel=Claude-3.5-Sonnet2026.02 | 0.85 | 0.13 | |
| w/o DefenseModel=Claude-3.5-Sonnet2026.02 | 0.88 | 0.1 |