Agent Safety Evaluation on Agent-SafetyBench aggregated clean and five attack types
26.31UBRSAFEHARNESS
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| SAFEHARNESSHarness=Multi-Agent, Model=DeepSeek-Chat2026.04 | 26.31 | 20.2 | 77.92 | 59.4 | 3.8 | 709 | |
| SAFEHARNESSHarness=ReAct, Model=DeepSeek-Chat2026.04 | 28.74 | 26.4 | 94.25 | 67.5 | 3.48 | 1,152 | |
| System-PromptHarness=Multi-Agent, Model=DeepSeek-Chat2026.04 | 31.88 | 29.2 | 91.75 | 62.3 | 3.14 | 0 | |
| SAFEHARNESSHarness=Multi-Agent, Model=DeepSeek-Reasoner2026.04 | 33.76 | 28.7 | 83.67 | 54.4 | 2.96 | 678 | |
| SAFEHARNESSHarness=ReAct, Model=DeepSeek-Reasoner2026.04 | 34.59 | 32.3 | 95.42 | 62.9 | 2.89 | 1,137 | |
| SAFEHARNESSHarness=Self-Evolving, Model=DeepSeek-Reasoner2026.04 | 38.14 | 36.9 | 94.83 | 57.5 | 2.62 | 1,181 | |
| SAFEHARNESSHarness=Self-Evolving, Model=DeepSeek-Chat2026.04 | 39.18 | 38 | 95.08 | 56.5 | 2.55 | 1,477 | |
| System-PromptHarness=Multi-Agent, Model=DeepSeek-Reasoner2026.04 | 39.25 | 35.5 | 91.5 | 55.2 | 2.55 | 0 | |
| LlamaFirewallHarness=Multi-Agent, Model=DeepSeek-Chat2026.04 | 39.73 | 33.9 | 86.83 | 54 | 2.52 | 148 | |
| System-PromptHarness=ReAct, Model=DeepSeek-Chat2026.04 | 40.67 | 38.7 | 94.67 | 55.7 | 2.46 | 0 | |
| GuardrailHarness=Multi-Agent, Model=DeepSeek-Chat2026.04 | 40.79 | 34.8 | 86.83 | 52.6 | 2.45 | 82 | |
| System-PromptHarness=ReAct, Model=DeepSeek-Reasoner2026.04 | 43.08 | 42.6 | 96.92 | 54.3 | 2.32 | 0 | |
| GuardrailHarness=Multi-Agent, Model=DeepSeek-Reasoner2026.04 | 44.8 | 41.8 | 90.58 | 48.5 | 2.23 | 54 | |
| UnprotectedHarness=Multi-Agent, Model=DeepSeek-Chat2026.04 | 44.99 | 39.4 | 86.5 | 48.1 | 2.22 | 0 | |
| LlamaFirewallHarness=Multi-Agent, Model=DeepSeek-Reasoner2026.04 | 45.99 | 42.8 | 91.33 | 48.6 | 2.17 | 96 | |
| LlamaFirewallHarness=ReAct, Model=DeepSeek-Chat2026.04 | 46.83 | 44.8 | 94.67 | 49.5 | 2.14 | 168 | |
| GuardrailHarness=ReAct, Model=DeepSeek-Chat2026.04 | 47.19 | 45 | 95 | 49.9 | 2.12 | 88 | |
| UnprotectedHarness=Multi-Agent, Model=DeepSeek-Reasoner2026.04 | 48.22 | 45.2 | 91.25 | 46 | 2.07 | 0 | |
| System-PromptHarness=Self-Evolving, Model=DeepSeek-Reasoner2026.04 | 49.52 | 48.4 | 96.42 | 47.7 | 2.02 | 0 | |
| UnprotectedHarness=ReAct, Model=DeepSeek-Chat2026.04 | 51.18 | 49.2 | 95.25 | 45.6 | 1.95 | 0 | |
| LlamaFirewallHarness=ReAct, Model=DeepSeek-Reasoner2026.04 | 51.26 | 49.5 | 95.75 | 46.4 | 1.95 | 184 | |
| GuardrailHarness=ReAct, Model=DeepSeek-Reasoner2026.04 | 52.81 | 51.3 | 96.42 | 44.7 | 1.89 | 67 | |
| System-PromptHarness=Self-Evolving, Model=DeepSeek-Chat2026.04 | 54.2 | 53 | 95.33 | 41.9 | 1.85 | 0 | |
| LlamaFirewallHarness=Self-Evolving, Model=DeepSeek-Reasoner2026.04 | 56.51 | 54.3 | 95.42 | 40.6 | 1.77 | 182 | |
| GuardrailHarness=Self-Evolving, Model=DeepSeek-Reasoner2026.04 | 56.54 | 55.7 | 96.25 | 40.5 | 1.77 | 75 | |
| LlamaFirewallHarness=Self-Evolving, Model=DeepSeek-Chat2026.04 | 56.72 | 55.3 | 94.92 | 39.3 | 1.76 | 215 | |
| UnprotectedHarness=ReAct, Model=DeepSeek-Reasoner2026.04 | 57.32 | 56.2 | 95.67 | 39.1 | 1.74 | 0 | |
| GuardrailHarness=Self-Evolving, Model=DeepSeek-Chat2026.04 | 59.26 | 56.9 | 94.92 | 37.4 | 1.69 | 99 | |
| UnprotectedHarness=Self-Evolving, Model=DeepSeek-Reasoner2026.04 | 61.83 | 61.6 | 95.42 | 33.6 | 1.62 | 0 | |
| UnprotectedHarness=Self-Evolving, Model=DeepSeek-Chat2026.04 | 61.98 | 60.9 | 95.33 | 33.9 | 1.61 | 0 |