LLM Safety Evaluation on HARMAMP (malicious instances)
86.36Harm ScoreVanilla LLM
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Vanilla LLMTarget Model=Llama-3.1-8B2026.06 | 86.36 | — | |
| Llama-Guard-3Target Model=Llama-3.1-8B, Protocol=prompt-only2026.06 | 61.36 | 50.48 | |
| Prompting-based MonitorTarget Model=Llama-3.1-8B2026.06 | 54.55 | 80 | |
| Llama-Guard-3Target Model=Llama-3.1-8B, Protocol=prompt-response2026.06 | 53.03 | 42.03 | |
| Llama-Guard-4Target Model=Llama-3.1-8B, Protocol=prompt-only2026.06 | 52.27 | 60.25 | |
| Llama-Guard-4Target Model=Llama-3.1-8B, Protocol=prompt-response2026.06 | 51.52 | 43.97 | |
| Vanilla LLMTarget Model=Qwen3-4B2026.06 | 42.42 | — | |
| SFT MonitorTarget Model=Llama-3.1-8B2026.06 | 40.91 | 70.45 | |
| GPT-oss-safeguardTarget Model=Llama-3.1-8B, Protocol=prompt-response2026.06 | 37.12 | 54.14 | |
| GPT-oss-safeguardTarget Model=Llama-3.1-8B, Protocol=prompt-only2026.06 | 29.55 | 76.04 | |
| Llama-Guard-3Target Model=Qwen3-4B, Protocol=prompt-only2026.06 | 28.03 | 28.99 | |
| Llama-Guard-3Target Model=Qwen3-4B, Protocol=prompt-response2026.06 | 24.24 | 14.7 | |
| Llama-Guard-4Target Model=Qwen3-4B, Protocol=prompt-only2026.06 | 22.73 | 34.38 | |
| Llama-Guard-4Target Model=Qwen3-4B, Protocol=prompt-response2026.06 | 21.97 | 18.39 | |
| Prompting-based MonitorTarget Model=Qwen3-4B2026.06 | 21.97 | 92.95 | |
| GPT-oss-safeguardTarget Model=Qwen3-4B, Protocol=prompt-response2026.06 | 19.7 | 24.07 | |
| SFT MonitorTarget Model=Qwen3-4B2026.06 | 15.91 | 66.79 | |
| GPT-oss-safeguardTarget Model=Qwen3-4B, Protocol=prompt-only2026.06 | 14.39 | 60.28 | |
| Vanilla LLMTarget Model=GPT-5-mini2026.06 | 14.39 | — | |
| Llama-Guard-3Target Model=GPT-5-mini, Protocol=prompt-only2026.06 | 12.88 | 19.08 | |
| Llama-Guard-3Target Model=GPT-5-mini, Protocol=prompt-response2026.06 | 12.12 | 2.14 | |
| Llama-Guard-4Target Model=GPT-5-mini, Protocol=prompt-response2026.06 | 12.12 | 4.47 | |
| GPT-oss-safeguardTarget Model=GPT-5-mini, Protocol=prompt-response2026.06 | 11.36 | 5.45 | |
| TRAJSAFETarget Model=Llama-3.1-8B2026.06 | 9.85 | 40.35 | |
| Prompting-based MonitorTarget Model=GPT-5-mini2026.06 | 9.85 | 95.31 | |
| Llama-Guard-4Target Model=GPT-5-mini, Protocol=prompt-only2026.06 | 9.09 | 30.57 | |
| GPT-oss-safeguardTarget Model=GPT-5-mini, Protocol=prompt-only2026.06 | 6.06 | 36.8 | |
| SFT MonitorTarget Model=GPT-5-mini2026.06 | 5.3 | 65.71 | |
| TRAJSAFETarget Model=Qwen3-4B2026.06 | 0.76 | 32.27 | |
| TRAJSAFETarget Model=GPT-5-mini2026.06 | 0.76 | 26.69 |