Tool Authorization Safety Evaluation on 8 Safety Scenarios 1.0 (test)
100Refusal RateFable 5
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Fable 52026.07 | 100 | 0 | 0 | 0 | 0.0088 | — | |
| MiniMax M32026.07 | 75 | 2 | 25 | 0 | 0.0005 | 0.006 | |
| MiMo V2.5 Pro2026.07 | 75 | 2 | 25 | 0 | 0.0005 | 0.012 | |
| GLM 5.22026.07 | 75 | 2 | 25 | 0 | 0.0006 | 0.018 | |
| Gemini 3.1 Pro2026.07 | 75 | 2 | 25 | 0 | 0.0037 | 0.012 | |
| Sonnet 52026.07 | 75 | 2 | 25 | 0 | 0.0046 | 0.018 | |
| GPT-5 mini2026.07 | 62 | 3 | 38 | 0 | 0.0006 | 0.013 | |
| Kimi K2.62026.07 | 62 | 3 | 38 | 0 | 0.0008 | 0.024 | |
| Nemotron 3 Ultra2026.07 | 62 | 3 | 38 | 0 | 0.001 | 0.03 | |
| Kimi K2.52026.07 | 50 | 4 | 50 | 0 | 0.0006 | 0.015 | |
| GPT-5.52026.07 | 50 | 4 | 50 | 0 | 0.0059 | 0.014 | |
| Opus 4.82026.07 | 50 | 4 | 50 | 0 | 0.0095 | 0.019 | |
| DeepSeek V4 Pro2026.07 | 38 | 5 | 62 | 0 | 0.0004 | 0.021 | |
| Gemini 3.5 Flash2026.07 | 38 | 5 | 62 | 0 | 0.0005 | 0.011 | |
| Qwen3 Maxdataset_subset=partial subset (four scenarios)2026.07 | 25 | 3 | 75 | 0 | 0.0005 | 0.012 |