Multi-turn Safety Risk Assessment on PostgreSQL
1ASRw/o Defense
Evaluation results
| Method | Links | ||
|---|---|---|---|
| w/o DefenseModel=Claude-3.5-Sonnet2026.02 | 1 | 0 | |
| FirewallModel=Claude-3.5-Sonnet2026.02 | 0.96 | 0.04 | |
| BaselineModel=Gemini-1.5-Flash2026.02 | 0.84 | 0.04 | |
| w/o DefenseModel=Gemini-1.5-Flash2026.02 | 0.72 | 0.16 | |
| FirewallModel=Gemini-1.5-Flash2026.02 | 0.68 | 0.2 | |
| ToolShieldModel=Gemini-1.5-Flash2026.02 | 0.6 | 0.4 | |
| BaselineModel=Claude-3.5-Sonnet2026.02 | 0.36 | 0.52 | |
| ToolShieldModel=Claude-3.5-Sonnet2026.02 | 0.12 | 0.88 |