# LLM Safety Evaluation on HARMAMP (malicious instances) Source: [LLM Safety Evaluation on HARMAMP (malicious instances)]() Collection index: [llms.txt]() Data updated: 2026-06-02T13:27:50.612Z Evaluation of LLM safety performance using the HARMAMP (malicious instances) dataset, measured by Harm Score and Intervention Turn Rate. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | LLM Safety Evaluation | | Dataset (including any stated split) | HARMAMP (malicious instances) | | Metric | Harm Score | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 30 of 30 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Harm Score](): Higher is better; unit: PERCENT. - [Intervention Turn Rate](): Higher is better; unit: PERCENT. ## Recorded results | Method | Harm Score | Result date | Report paper | | --- | --- | --- | --- | | Vanilla LLM | 86.36 | Not recorded | 6a1e6426dbd7150506f85805 | | Llama-Guard-3 | 61.36 | Not recorded | 6a1e6426dbd7150506f85805 | | Prompting-based Monitor | 54.55 | Not recorded | 6a1e6426dbd7150506f85805 | | Llama-Guard-3 | 53.03 | Not recorded | 6a1e6426dbd7150506f85805 | | Llama-Guard-4 | 52.27 | Not recorded | 6a1e6426dbd7150506f85805 | | Llama-Guard-4 | 51.52 | Not recorded | 6a1e6426dbd7150506f85805 | | Vanilla LLM | 42.42 | Not recorded | 6a1e6426dbd7150506f85805 | | SFT Monitor | 40.91 | Not recorded | 6a1e6426dbd7150506f85805 | | GPT-oss-safeguard | 37.12 | Not recorded | 6a1e6426dbd7150506f85805 | | GPT-oss-safeguard | 29.55 | Not recorded | 6a1e6426dbd7150506f85805 | | Llama-Guard-3 | 28.03 | Not recorded | 6a1e6426dbd7150506f85805 | | Llama-Guard-3 | 24.24 | Not recorded | 6a1e6426dbd7150506f85805 | | Llama-Guard-4 | 22.73 | Not recorded | 6a1e6426dbd7150506f85805 | | Llama-Guard-4 | 21.97 | Not recorded | 6a1e6426dbd7150506f85805 | | Prompting-based Monitor | 21.97 | Not recorded | 6a1e6426dbd7150506f85805 | | GPT-oss-safeguard | 19.7 | Not recorded | 6a1e6426dbd7150506f85805 | | SFT Monitor | 15.91 | Not recorded | 6a1e6426dbd7150506f85805 | | GPT-oss-safeguard | 14.39 | Not recorded | 6a1e6426dbd7150506f85805 | | Vanilla LLM | 14.39 | Not recorded | 6a1e6426dbd7150506f85805 | | Llama-Guard-3 | 12.88 | Not recorded | 6a1e6426dbd7150506f85805 | | Llama-Guard-3 | 12.12 | Not recorded | 6a1e6426dbd7150506f85805 | | Llama-Guard-4 | 12.12 | Not recorded | 6a1e6426dbd7150506f85805 | | GPT-oss-safeguard | 11.36 | Not recorded | 6a1e6426dbd7150506f85805 | | TRAJSAFE | 9.85 | Not recorded | 6a1e6426dbd7150506f85805 | | Prompting-based Monitor | 9.85 | Not recorded | 6a1e6426dbd7150506f85805 | | Llama-Guard-4 | 9.09 | Not recorded | 6a1e6426dbd7150506f85805 | | GPT-oss-safeguard | 6.06 | Not recorded | 6a1e6426dbd7150506f85805 | | SFT Monitor | 5.3 | Not recorded | 6a1e6426dbd7150506f85805 | | TRAJSAFE | 0.76 | Not recorded | 6a1e6426dbd7150506f85805 | | TRAJSAFE | 0.76 | Not recorded | 6a1e6426dbd7150506f85805 | ### Vanilla LLM Harm Score: 86.36. Result ID: 6a1ed9e9fbc373592c1aea04. [Reporting paper]() Recorded evaluation configuration: {"Target Model":"Llama-3.1-8B"}. This record does not assert independent reproduction. ### Llama-Guard-3 Harm Score: 61.36. Result ID: 6a1ed9e9fbc373592c1aea05. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Target Model":"Llama-3.1-8B","Protocol":"prompt-only"}. This record does not assert independent reproduction. ### Prompting-based Monitor Harm Score: 54.55. Result ID: 6a1ed9e9fbc373592c1aea0b. [Reporting paper]() Recorded evaluation configuration: {"Target Model":"Llama-3.1-8B"}. This record does not assert independent reproduction. ### Llama-Guard-3 Harm Score: 53.03. Result ID: 6a1ed9e9fbc373592c1aea06. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Target Model":"Llama-3.1-8B","Protocol":"prompt-response"}. This record does not assert independent reproduction. ### Llama-Guard-4 Harm Score: 52.27. Result ID: 6a1ed9e9fbc373592c1aea07. [Reporting paper]() Recorded evaluation configuration: {"Target Model":"Llama-3.1-8B","Protocol":"prompt-only"}. This record does not assert independent reproduction. ### Llama-Guard-4 Harm Score: 51.52. Result ID: 6a1ed9e9fbc373592c1aea08. [Reporting paper]() Recorded evaluation configuration: {"Target Model":"Llama-3.1-8B","Protocol":"prompt-response"}. This record does not assert independent reproduction. ### Vanilla LLM Harm Score: 42.42. Result ID: 6a1ed9e9fbc373592c1aea0e. [Reporting paper]() Recorded evaluation configuration: {"Target Model":"Qwen3-4B"}. This record does not assert independent reproduction. ### SFT Monitor Harm Score: 40.91. Result ID: 6a1ed9e9fbc373592c1aea0c. [Reporting paper]() Recorded evaluation configuration: {"Target Model":"Llama-3.1-8B"}. This record does not assert independent reproduction. ### GPT-oss-safeguard Harm Score: 37.12. Result ID: 6a1ed9e9fbc373592c1aea0a. [Reporting paper]() Recorded evaluation configuration: {"Target Model":"Llama-3.1-8B","Protocol":"prompt-response"}. This record does not assert independent reproduction. ### GPT-oss-safeguard Harm Score: 29.55. Result ID: 6a1ed9e9fbc373592c1aea09. [Reporting paper]() Recorded evaluation configuration: {"Target Model":"Llama-3.1-8B","Protocol":"prompt-only"}. This record does not assert independent reproduction. ### Llama-Guard-3 Harm Score: 28.03. Result ID: 6a1ed9e9fbc373592c1aea0f. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Target Model":"Qwen3-4B","Protocol":"prompt-only"}. This record does not assert independent reproduction. ### Llama-Guard-3 Harm Score: 24.24. Result ID: 6a1ed9e9fbc373592c1aea10. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Target Model":"Qwen3-4B","Protocol":"prompt-response"}. This record does not assert independent reproduction. ### Llama-Guard-4 Harm Score: 22.73. Result ID: 6a1ed9e9fbc373592c1aea11. [Reporting paper]() Recorded evaluation configuration: {"Target Model":"Qwen3-4B","Protocol":"prompt-only"}. This record does not assert independent reproduction. ### Llama-Guard-4 Harm Score: 21.97. Result ID: 6a1ed9e9fbc373592c1aea12. [Reporting paper]() Recorded evaluation configuration: {"Target Model":"Qwen3-4B","Protocol":"prompt-response"}. This record does not assert independent reproduction. ### Prompting-based Monitor Harm Score: 21.97. Result ID: 6a1ed9e9fbc373592c1aea15. [Reporting paper]() Recorded evaluation configuration: {"Target Model":"Qwen3-4B"}. This record does not assert independent reproduction. ### GPT-oss-safeguard Harm Score: 19.7. Result ID: 6a1ed9e9fbc373592c1aea14. [Reporting paper]() Recorded evaluation configuration: {"Target Model":"Qwen3-4B","Protocol":"prompt-response"}. This record does not assert independent reproduction. ### SFT Monitor Harm Score: 15.91. Result ID: 6a1ed9e9fbc373592c1aea16. [Reporting paper]() Recorded evaluation configuration: {"Target Model":"Qwen3-4B"}. This record does not assert independent reproduction. ### GPT-oss-safeguard Harm Score: 14.39. Result ID: 6a1ed9e9fbc373592c1aea13. [Reporting paper]() Recorded evaluation configuration: {"Target Model":"Qwen3-4B","Protocol":"prompt-only"}. This record does not assert independent reproduction. ### Vanilla LLM Harm Score: 14.39. Result ID: 6a1ed9e9fbc373592c1aea18. [Reporting paper]() Recorded evaluation configuration: {"Target Model":"GPT-5-mini"}. This record does not assert independent reproduction. ### Llama-Guard-3 Harm Score: 12.88. Result ID: 6a1ed9e9fbc373592c1aea19. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Target Model":"GPT-5-mini","Protocol":"prompt-only"}. This record does not assert independent reproduction. ### Llama-Guard-3 Harm Score: 12.12. Result ID: 6a1ed9e9fbc373592c1aea1a. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Target Model":"GPT-5-mini","Protocol":"prompt-response"}. This record does not assert independent reproduction. ### Llama-Guard-4 Harm Score: 12.12. Result ID: 6a1ed9e9fbc373592c1aea1c. [Reporting paper]() Recorded evaluation configuration: {"Target Model":"GPT-5-mini","Protocol":"prompt-response"}. This record does not assert independent reproduction. ### GPT-oss-safeguard Harm Score: 11.36. Result ID: 6a1ed9e9fbc373592c1aea1e. [Reporting paper]() Recorded evaluation configuration: {"Target Model":"GPT-5-mini","Protocol":"prompt-response"}. This record does not assert independent reproduction. ### TRAJSAFE Harm Score: 9.85. Result ID: 6a1ed9e9fbc373592c1aea0d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Target Model":"Llama-3.1-8B"}. This record does not assert independent reproduction. ### Prompting-based Monitor Harm Score: 9.85. Result ID: 6a1ed9e9fbc373592c1aea1f. [Reporting paper]() Recorded evaluation configuration: {"Target Model":"GPT-5-mini"}. This record does not assert independent reproduction. ### Llama-Guard-4 Harm Score: 9.09. Result ID: 6a1ed9e9fbc373592c1aea1b. [Reporting paper]() Recorded evaluation configuration: {"Target Model":"GPT-5-mini","Protocol":"prompt-only"}. This record does not assert independent reproduction. ### GPT-oss-safeguard Harm Score: 6.06. Result ID: 6a1ed9e9fbc373592c1aea1d. [Reporting paper]() Recorded evaluation configuration: {"Target Model":"GPT-5-mini","Protocol":"prompt-only"}. This record does not assert independent reproduction. ### SFT Monitor Harm Score: 5.3. Result ID: 6a1ed9e9fbc373592c1aea20. [Reporting paper]() Recorded evaluation configuration: {"Target Model":"GPT-5-mini"}. This record does not assert independent reproduction. ### TRAJSAFE Harm Score: 0.76. Result ID: 6a1ed9e9fbc373592c1aea17. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Target Model":"Qwen3-4B"}. This record does not assert independent reproduction. ### TRAJSAFE Harm Score: 0.76. Result ID: 6a1ed9e9fbc373592c1aea21. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Target Model":"GPT-5-mini"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.