Agent Safety Evaluation on AgentHarm Libra
83ScoreSafeMCP
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SafeMCPBase Model=GPT-4o2026.06 | 83 | 66 | |
| SafeMCPBase Model=GPT-4o-mini2026.06 | 78 | 55 | |
| RL-GuardBase Model=GPT-4o2026.06 | 75 | 59 | |
| GPT-4oBase Model=GPT-4o2026.06 | 72 | 60 | |
| Llama Guard 3Base Model=GPT-4o2026.06 | 72 | 58 | |
| SafironBase Model=GPT-4o2026.06 | 72 | 60 | |
| GPT-4o-miniBase Model=GPT-4o-mini2026.06 | 72 | 52 | |
| RL-GuardBase Model=GPT-4o-mini2026.06 | 72 | 54 | |
| Llama Guard 3Base Model=GPT-4o-mini2026.06 | 70 | 54 | |
| SafironBase Model=GPT-4o-mini2026.06 | 69 | 51 | |
| LLaMA-3.1-Instruct-8BBase Model=LLaMA-3.1-Instruct-8B2026.06 | 67 | 37 | |
| NeMoGuard-8bBase Model=GPT-4o2026.06 | 66 | 53 | |
| SafironBase Model=LLaMA-3.1-Instruct-8B2026.06 | 66 | 38 | |
| SafeMCPBase Model=LLaMA-3.1-Instruct-8B2026.06 | 66 | 35 | |
| NeMoGuard-8bBase Model=GPT-4o-mini2026.06 | 64 | 50 | |
| RL-GuardBase Model=LLaMA-3.1-Instruct-8B2026.06 | 64 | 33 | |
| Llama Guard 3Base Model=LLaMA-3.1-Instruct-8B2026.06 | 58 | 35 | |
| NeMoGuard-8bBase Model=LLaMA-3.1-Instruct-8B2026.06 | 58 | 35 | |
| AgentMonitorBase Model=GPT-4o2026.06 | 55 | 48 | |
| AgentMonitorBase Model=GPT-4o-mini2026.06 | 54 | 45 | |
| Qwen3Guard-8BBase Model=GPT-4o2026.06 | 50 | 44 | |
| Qwen3Guard-8BBase Model=GPT-4o-mini2026.06 | 49 | 43 | |
| AgentMonitorBase Model=LLaMA-3.1-Instruct-8B2026.06 | 43 | 32 | |
| Qwen3Guard-8BBase Model=LLaMA-3.1-Instruct-8B2026.06 | 41 | 32 | |
| Lakera-ChainGuardBase Model=GPT-4o2026.06 | 40 | 34 | |
| Lakera-ChainGuardBase Model=GPT-4o-mini2026.06 | 39 | 33 | |
| Lakera-ChainGuardBase Model=LLaMA-3.1-Instruct-8B2026.06 | 36 | 30 |