Multi-turn Safety Risk Assessment on Playwright tasks
80ASRw/o Defense
Evaluation results
| Method | Links | ||
|---|---|---|---|
| w/o DefenseModel=Claude-3.5-Sonnet2026.02 | 80 | 16 | |
| FirewallModel=Claude-3.5-Sonnet2026.02 | 80 | 16 | |
| w/o DefenseModel=Gemini-1.5-Flash2026.02 | 76 | 8 | |
| FirewallModel=Gemini-1.5-Flash2026.02 | 76 | 8 | |
| BaselineModel=Gemini-1.5-Flash2026.02 | 72 | 12 | |
| ToolShieldModel=Gemini-1.5-Flash2026.02 | 60 | 16 | |
| BaselineModel=Claude-3.5-Sonnet2026.02 | 40 | 56 | |
| ToolShieldModel=Claude-3.5-Sonnet2026.02 | 4 | 92 |