Tool Authorization Safety Evaluation on 8 Safety Scenarios 1.0 (test) benchmark leaderboard · SOTA2 Research

Tool Authorization Safety Evaluation on 8 Safety Scenarios 1.0 (test)

100Refusal RateFable 5

Evaluation Results

Method	Links
Fable 52026.07		100	0	0	0	0.0088	—
MiniMax M32026.07		75	2	25	0	0.0005	0.006
MiMo V2.5 Pro2026.07		75	2	25	0	0.0005	0.012
GLM 5.22026.07		75	2	25	0	0.0006	0.018
Gemini 3.1 Pro2026.07		75	2	25	0	0.0037	0.012
Sonnet 52026.07		75	2	25	0	0.0046	0.018
GPT-5 mini2026.07		62	3	38	0	0.0006	0.013
Kimi K2.62026.07		62	3	38	0	0.0008	0.024
Nemotron 3 Ultra2026.07		62	3	38	0	0.001	0.03
Kimi K2.52026.07		50	4	50	0	0.0006	0.015
GPT-5.52026.07		50	4	50	0	0.0059	0.014
Opus 4.82026.07		50	4	50	0	0.0095	0.019
DeepSeek V4 Pro2026.07		38	5	62	0	0.0004	0.021
Gemini 3.5 Flash2026.07		38	5	62	0	0.0005	0.011
Qwen3 Max2026.07		25	3	75	0	0.0005	0.012