Over-refusal evaluation on XS (test)
22OR ScoreTHRD-style
Evaluation results
| Method | Links | |
|---|---|---|
| THRD-styleDefense Type=Trajectory-aware firewalls, Target Model=mistral, Judge Model=qwen3.5:122b2026.07 | 22 | |
| Granite Guardian 8BDefense Type=Per-message guard models, Target Model=mistral, Judge Model=qwen3.5:122b2026.07 | 21 | |
| ShieldGemma 9BDefense Type=Per-message guard models, Target Model=mistral, Judge Model=qwen3.5:122b2026.07 | 18 | |
| TCA-styleDefense Type=Trajectory-aware firewalls, Target Model=mistral, Judge Model=qwen3.5:122b2026.07 | 14 | |
| Cognitive FirewallDefense Type=Trajectory-aware firewalls, Target Model=mistral, Judge Model=qwen3.5:122b2026.07 | 8 | |
| Llama Guard 3 8BDefense Type=Per-message guard models, Target Model=mistral, Judge Model=qwen3.5:122b2026.07 | 3 | |
| none (undefended)Defense Type=undefended, Target Model=mistral, Judge Model=qwen3.5:122b2026.07 | 0 |