Agent Safety Evaluation on ToolEmu benchmark leaderboard · SOTA2 Research

Agent Safety Evaluation on ToolEmu

99SafetySafeMCP

Evaluation Results

Method	Links
SafeMCP2026.06		99	22	60	44
SafeMCP2026.06		98	15	56	40
RL-Guard2026.06		94	39	67	57
SafeMCP2026.06		94	42	68	59
RL-Guard2026.06		89	9	49	35
AgentMonitor2026.06		89	28	59	48
RL-Guard2026.06		88	9	49	35
AgentMonitor2026.06		88	3	45	31
SafeMCP2026.06		85	7	46	33
AgentMonitor2026.06		83	10	47	35
AgentMonitor2026.06		81	22	52	43
ChainGuard2026.06		77	40	59	55
RL-Guard2026.06		77	3	40	30
NeMoGuard-8B2026.06		76	43	60	56
Qwen3Guard-8B2026.06		72	43	57	55
ChainGuard2026.06		71	15	43	36
Llama Guard 32026.06		69	47	58	57
ChainGuard2026.06		65	11	38	32
Safiron2026.06		65	53	59	59
Qwen3Guard-8B2026.06		63	21	42	38
w/o defense2026.06		63	54	58	58
ChainGuard2026.06		60	5	33	27
NeMoGuard-8B2026.06		58	22	40	37
Qwen3Guard-8B2026.06		58	15	36	33
NeMoGuard-8B2026.06		52	7	30	26
Llama Guard 32026.06		51	7	29	26
Qwen3Guard-8B2026.06		49	6	28	25
Safiron2026.06		45	24	34	33
Safiron2026.06		45	17	31	29
Llama Guard 32026.06		44	17	31	29
NeMoGuard-8B2026.06		43	14	29	27
w/o defense2026.06		42	25	34	33
w/o defense2026.06		42	17	30	28
Llama Guard 32026.06		41	20	31	30
Safiron2026.06		36	7	22	20
w/o defense2026.06		33	7	20	19