Mental Health Safety Evaluation on MHSafeEval (v1.0)
37.5ASR (Gaslighting)MiniMax m2.5
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| MiniMax m2.5Mutation Iteration=Disabled2026.04 | 37.5 | 50 | 28.6 | 22.2 | 77.8 | 87.5 | 58.8 | 52.9 | 15.7 | 61.4 | |
| Llama 3.1Mutation Iteration=Disabled2026.04 | 54.3 | 50 | 21.6 | 27.8 | 81.4 | 96.6 | 75 | 58.9 | 57 | 97.3 | |
| GPT-3.5Mutation Iteration=Disabled2026.04 | 60 | 50.9 | 39.3 | 27.6 | 70.8 | 100 | 72.7 | 60.3 | 38.4 | 99.5 | |
| MiMoMutation Iteration=Disabled2026.04 | 71.9 | 54.3 | 27 | 57.6 | 90.6 | 95 | 71.2 | 64.9 | 38.9 | 98.9 | |
| Gemini 2.5Mutation Iteration=Disabled2026.04 | 79.6 | 68.8 | 47.5 | 63.5 | 90.7 | 100 | 59.6 | 70.8 | 19.5 | 99.2 | |
| DeepSeek v3.2Mutation Iteration=Disabled2026.04 | 80.9 | 77.8 | 59.5 | 59.5 | 93.8 | 94.3 | 73.8 | 76.2 | 20 | 99.5 | |
| Haiku 4.5Mutation Iteration=Disabled2026.04 | 88.2 | 87.8 | 60.7 | 46.9 | 94.2 | 86.2 | 87.1 | 78.9 | 83.2 | 99.5 | |
| Gemma 4Mutation Iteration=Disabled2026.04 | 90 | 94.7 | 66.7 | 70.9 | 100 | 100 | 84 | 87.3 | 15.7 | 97.8 | |
| MiniMax m2.5Mutation Iteration=Enabled2026.04 | 93.3 | 86 | 81.5 | 83.8 | 92.9 | 98 | 95.5 | 91.4 | 3 | 81.1 | |
| Gemini 2.5Mutation Iteration=Enabled2026.04 | 93.8 | 98.3 | 92.6 | 98.3 | 98.2 | 100 | 95.9 | 97 | 3.8 | 97.3 | |
| DeepSeek v3.2Mutation Iteration=Enabled2026.04 | 93.8 | 97 | 94.4 | 95.7 | 100 | 100 | 96.7 | 97 | 12.4 | 99.7 | |
| Llama 3.1Mutation Iteration=Enabled2026.04 | 94.5 | 87.5 | 79.3 | 83.8 | 89.7 | 98.8 | 98.3 | 92.2 | 55.7 | 94.1 | |
| MiMoMutation Iteration=Enabled2026.04 | 95.2 | 96 | 93.9 | 92.8 | 95.6 | 96.4 | 90.9 | 94.3 | 34.3 | 99.7 | |
| Haiku 4.5Mutation Iteration=Enabled2026.04 | 98.3 | 100 | 96.9 | 91.7 | 98.2 | 95.9 | 97.1 | 97 | 85.9 | 98.6 | |
| GPT-3.5Mutation Iteration=Enabled2026.04 | 100 | 100 | 66.7 | 75 | 100 | 100 | 100 | 94.3 | 7.1 | 100 | |
| Gemma 4Mutation Iteration=Enabled2026.04 | 100 | 97.7 | 100 | 100 | 100 | 100 | 100 | 99.7 | 7 | 95.9 |