LLM Safety Evaluation on Aymara Risk and Responsibility Matrix 10 Domains
86.2Mean Safety ScoreClaude Haiku 3.5
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Claude Haiku 3.5Provider=Anthropic2025.07 | 86.2 | 21.1 | 97.8 | 45.5 | 100 | |
| Claude Sonnet 4Provider=Anthropic2025.07 | 86 | 24.5 | 98 | 31.8 | 100 | |
| Nova PremierProvider=Amazon2025.07 | 85.6 | 24.5 | 97.8 | 31.8 | 100 | |
| GPT 4.1Provider=OpenAI2025.07 | 85.5 | 21.4 | 95.5 | 31.8 | 100 | |
| GPT 4.1 NanoProvider=OpenAI2025.07 | 85.3 | 22.4 | 97.7 | 31.8 | 100 | |
| GPT 4.1 MiniProvider=OpenAI2025.07 | 83.1 | 23.3 | 93.5 | 31.8 | 100 | |
| Nova LiteProvider=Amazon2025.07 | 78.8 | 26.1 | 86.4 | 22.7 | 100 | |
| Gemini 2.5 FlashProvider=Google2025.07 | 78.5 | 22.5 | 85.3 | 40.9 | 100 | |
| Nova ProProvider=Amazon2025.07 | 77.4 | 26.2 | 84.6 | 27.3 | 100 | |
| Gemini 2.5 ProProvider=Google2025.07 | 75.5 | 28.2 | 87 | 22.7 | 100 | |
| Mistral SmallProvider=Mistral AI2025.07 | 70.3 | 23.7 | 73.2 | 9.1 | 90.9 | |
| Mistral Large 2Provider=Mistral AI2025.07 | 69.7 | 23.1 | 75.9 | 13.6 | 90.9 | |
| Grok 3Provider=xAI2025.07 | 68.7 | 18.8 | 74.4 | 40.9 | 95.5 | |
| Llama 4 MaverickProvider=Meta2025.07 | 67.4 | 22.5 | 71.1 | 22.7 | 100 | |
| Jamba 1.5 MiniProvider=AI21 Labs2025.07 | 65 | 27.9 | 68.2 | 9.1 | 95.5 | |
| DeepSeek-R1Provider=DeepSeek2025.07 | 64.9 | 25.9 | 72.7 | 20 | 95.5 | |
| Llama 4 ScoutProvider=Meta2025.07 | 64.2 | 24.8 | 68.2 | 13.6 | 90.9 | |
| Jamba 1.5 LargeProvider=AI21 Labs2025.07 | 60.8 | 28.2 | 61.4 | 18.2 | 95.5 | |
| Command R+Provider=Cohere2025.07 | 59.5 | 34.1 | 69 | 8.7 | 100 | |
| Command RProvider=Cohere2025.07 | 52.4 | 30.2 | 47.7 | 8.7 | 90.9 |