Benign tool-calling reliability on AgentHarm Benign
0Refusal RateGPT-4o
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GPT-4oSafety Scaffolding=Base, Evaluation Mode=Zero-shot2026.03 | 0 | 0.8 | |
| GPT-5Safety Scaffolding=Base, Evaluation Mode=Zero-shot2026.03 | 0 | 0.68 | |
| Qwen2.5-7BSafety Scaffolding=Base, Evaluation Mode=Zero-shot2026.03 | 13 | 0.51 | |
| Qwen3-4B-ThinkSafety Scaffolding=Base, Evaluation Mode=Zero-shot2026.03 | 13 | 0.66 | |
| Qwen2.5-7BSafety Scaffolding=MOSAIC, Evaluation Mode=Zero-shot2026.03 | 15 | 0.61 | |
| GPT-4oSafety Scaffolding=MOSAIC, Evaluation Mode=Zero-shot2026.03 | 19 | 0.75 | |
| Phi-4Safety Scaffolding=MOSAIC, Evaluation Mode=Zero-shot2026.03 | 19 | 0.75 | |
| Qwen3-4B-ThinkSafety Scaffolding=MOSAIC, Evaluation Mode=Zero-shot2026.03 | 23 | 0.7 | |
| GPT-5Safety Scaffolding=MOSAIC, Evaluation Mode=Zero-shot2026.03 | 24 | 0.73 | |
| Phi-4Safety Scaffolding=Base, Evaluation Mode=Zero-shot2026.03 | 43 | 0.77 |