# Mental Health Safety Evaluation on MHSafeEval (v1.0) Source: [Mental Health Safety Evaluation on MHSafeEval (v1.0)]() Collection index: [llms.txt]() Data updated: 2026-04-21T11:46:08.118Z Evaluation of model safety concerning seven specific mental health harm categories using the MHSafeEval 1.0 dataset, focusing on Attack Success Rates (ASR) across categories and overall refusal/comprehension rates. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Mental Health Safety Evaluation | | Dataset (including any stated split) | MHSafeEval seven mental-health harm categories 1.0 | | Metric | ASR (Gaslighting) | | Unit | PERCENT | | Direction | Lower is better | | Results loaded | 16 of 16 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [ASR (Gaslighting)](): Lower is better; unit: PERCENT. - [ASR (Blaming)](): Lower is better; unit: PERCENT. - [ASR (Toxic Lang.)](): Lower is better; unit: PERCENT. - [ASR (Nonfactual)](): Lower is better; unit: PERCENT. - [ASR (Overpath.)](): Lower is better; unit: PERCENT. - [ASR (Dep. Ind.)](): Lower is better; unit: PERCENT. - [ASR (Invalidation)](): Lower is better; unit: PERCENT. - [ASR (Overall)](): Lower is better; unit: PERCENT. - [Refusal Rate (RR)](): Higher is better; unit: PERCENT. - [Comprehension (Cmp.)](): Higher is better; unit: PERCENT. ## Recorded results | Method | ASR (Gaslighting) | Result date | Report paper | | --- | --- | --- | --- | | MiniMax m2.5 | 37.5 | Not recorded | 69e705027d0172140ec7f19b | | Llama 3.1 | 54.3 | Not recorded | 69e705027d0172140ec7f19b | | GPT-3.5 | 60 | Not recorded | 69e705027d0172140ec7f19b | | MiMo | 71.9 | Not recorded | 69e705027d0172140ec7f19b | | Gemini 2.5 | 79.6 | Not recorded | 69e705027d0172140ec7f19b | | DeepSeek v3.2 | 80.9 | Not recorded | 69e705027d0172140ec7f19b | | Haiku 4.5 | 88.2 | Not recorded | 69e705027d0172140ec7f19b | | Gemma 4 | 90 | Not recorded | 69e705027d0172140ec7f19b | | MiniMax m2.5 | 93.3 | Not recorded | 69e705027d0172140ec7f19b | | Gemini 2.5 | 93.8 | Not recorded | 69e705027d0172140ec7f19b | | DeepSeek v3.2 | 93.8 | Not recorded | 69e705027d0172140ec7f19b | | Llama 3.1 | 94.5 | Not recorded | 69e705027d0172140ec7f19b | | MiMo | 95.2 | Not recorded | 69e705027d0172140ec7f19b | | Haiku 4.5 | 98.3 | Not recorded | 69e705027d0172140ec7f19b | | GPT-3.5 | 100 | Not recorded | 69e705027d0172140ec7f19b | | Gemma 4 | 100 | Not recorded | 69e705027d0172140ec7f19b | ### MiniMax m2.5 ASR (Gaslighting): 37.5. Result ID: 69e7632347f34ce924cec0e4. [Reporting paper]() Recorded evaluation configuration: {"Mutation Iteration":"Disabled"}. This record does not assert independent reproduction. ### Llama 3.1 ASR (Gaslighting): 54.3. Result ID: 69e7632347f34ce924cec0da. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Mutation Iteration":"Disabled"}. This record does not assert independent reproduction. ### GPT-3.5 ASR (Gaslighting): 60. Result ID: 69e7632347f34ce924cec0d8. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Mutation Iteration":"Disabled"}. This record does not assert independent reproduction. ### MiMo ASR (Gaslighting): 71.9. Result ID: 69e7632347f34ce924cec0e6. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Mutation Iteration":"Disabled"}. This record does not assert independent reproduction. ### Gemini 2.5 ASR (Gaslighting): 79.6. Result ID: 69e7632347f34ce924cec0dc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Mutation Iteration":"Disabled"}. This record does not assert independent reproduction. ### DeepSeek v3.2 ASR (Gaslighting): 80.9. Result ID: 69e7632347f34ce924cec0e0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Mutation Iteration":"Disabled"}. This record does not assert independent reproduction. ### Haiku 4.5 ASR (Gaslighting): 88.2. Result ID: 69e7632347f34ce924cec0de. [Reporting paper]() Recorded evaluation configuration: {"Mutation Iteration":"Disabled"}. This record does not assert independent reproduction. ### Gemma 4 ASR (Gaslighting): 90. Result ID: 69e7632347f34ce924cec0e2. [Reporting paper]() Recorded evaluation configuration: {"Mutation Iteration":"Disabled"}. This record does not assert independent reproduction. ### MiniMax m2.5 ASR (Gaslighting): 93.3. Result ID: 69e7632347f34ce924cec0e3. [Reporting paper]() Recorded evaluation configuration: {"Mutation Iteration":"Enabled"}. This record does not assert independent reproduction. ### Gemini 2.5 ASR (Gaslighting): 93.8. Result ID: 69e7632347f34ce924cec0db. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Mutation Iteration":"Enabled"}. This record does not assert independent reproduction. ### DeepSeek v3.2 ASR (Gaslighting): 93.8. Result ID: 69e7632347f34ce924cec0df. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Mutation Iteration":"Enabled"}. This record does not assert independent reproduction. ### Llama 3.1 ASR (Gaslighting): 94.5. Result ID: 69e7632347f34ce924cec0d9. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Mutation Iteration":"Enabled"}. This record does not assert independent reproduction. ### MiMo ASR (Gaslighting): 95.2. Result ID: 69e7632347f34ce924cec0e5. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Mutation Iteration":"Enabled"}. This record does not assert independent reproduction. ### Haiku 4.5 ASR (Gaslighting): 98.3. Result ID: 69e7632347f34ce924cec0dd. [Reporting paper]() Recorded evaluation configuration: {"Mutation Iteration":"Enabled"}. This record does not assert independent reproduction. ### GPT-3.5 ASR (Gaslighting): 100. Result ID: 69e7632347f34ce924cec0d7. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Mutation Iteration":"Enabled"}. This record does not assert independent reproduction. ### Gemma 4 ASR (Gaslighting): 100. Result ID: 69e7632347f34ce924cec0e1. [Reporting paper]() Recorded evaluation configuration: {"Mutation Iteration":"Enabled"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.