LLM Agent Utility on Frontier Models Evaluation Set Benign Queries
1.1Benign Refusal RateQwen3-32B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Qwen3-32BDefense=Base2026.05 | 1.1 | 82.1 | |
| GPT-5Defense=Base2026.05 | 2.8 | 69.4 | |
| GPT-5 + SAFEHARBORDefense=SAFEHARBOR2026.05 | 5.6 | 68.1 | |
| Claude-3.5-Sonnet + SAFEHARBORDefense=SAFEHARBOR2026.05 | 14.8 | 62.1 | |
| Qwen3-32B + SAFEHARBORDefense=SAFEHARBOR2026.05 | 17.6 | 65.7 | |
| Claude-3.5-SonnetDefense=Base2026.05 | 18.7 | 59.7 |