Safety Classification Disagreement on WildChat Intent
0.4Disagreement Rate (per 1k conv)Gemini 3.1
Evaluation Results
| Method | Links | |
|---|---|---|
| Gemini 3.1Category=Hate Speech, Evaluation Protocol=Constitution2026.05 | 0.4 | |
| GPT-5.4Category=Hate Speech, Evaluation Protocol=Constitution2026.05 | 0.4 | |
| GPT-5.4 MiniCategory=Harassment, Evaluation Protocol=Constitution2026.05 | 0.4 | |
| GPT-5.4 MiniCategory=Hate Speech, Evaluation Protocol=Constitution2026.05 | 0.6 | |
| Gemini 3.1Category=Harassment, Evaluation Protocol=Constitution2026.05 | 0.7 | |
| Safeguard 20BCategory=Hate Speech, Evaluation Protocol=Constitution2026.05 | 0.9 | |
| GPT-5.4Category=Harassment, Evaluation Protocol=Constitution2026.05 | 1 | |
| Gemini 3.1Category=Non-Violent Crime, Evaluation Protocol=Constitution2026.05 | 1.4 | |
| Gemini 3.1Category=Hate Speech, Evaluation Protocol=Definition2026.05 | 1.5 | |
| GPT-5.4Category=Non-Violent Crime, Evaluation Protocol=Constitution2026.05 | 1.6 | |
| Safeguard 20BCategory=Non-Violent Crime, Evaluation Protocol=Constitution2026.05 | 1.8 | |
| Safeguard 20BCategory=Hate Speech, Evaluation Protocol=Definition2026.05 | 1.9 | |
| GPT-5.4 NanoCategory=Hate Speech, Evaluation Protocol=Constitution2026.05 | 2.2 | |
| GPT-5.4Category=Hate Speech, Evaluation Protocol=Definition2026.05 | 2.7 | |
| GPT-5.4 MiniCategory=Non-Violent Crime, Evaluation Protocol=Constitution2026.05 | 2.8 | |
| Safeguard 20BCategory=Harassment, Evaluation Protocol=Constitution2026.05 | 2.8 | |
| GPT-5.4 MiniCategory=Hate Speech, Evaluation Protocol=Definition2026.05 | 2.9 | |
| GPT-5.4 NanoCategory=Non-Violent Crime, Evaluation Protocol=Constitution2026.05 | 5.1 | |
| GPT-5.4 NanoCategory=Hate Speech, Evaluation Protocol=Definition2026.05 | 5.6 | |
| GPT-5.4 NanoCategory=Harassment, Evaluation Protocol=Constitution2026.05 | 5.9 | |
| Gemini 3.1Category=Non-Violent Crime, Evaluation Protocol=Definition2026.05 | 6.2 | |
| GPT-5.4 MiniCategory=Non-Violent Crime, Evaluation Protocol=Definition2026.05 | 7.2 | |
| GPT-5.4Category=Non-Violent Crime, Evaluation Protocol=Definition2026.05 | 10.1 | |
| Gemini 3.1Category=Harassment, Evaluation Protocol=Definition2026.05 | 11.5 | |
| Safeguard 20BCategory=Non-Violent Crime, Evaluation Protocol=Definition2026.05 | 11.5 | |
| Safeguard 20BCategory=Harassment, Evaluation Protocol=Definition2026.05 | 15.3 | |
| GPT-5.4 NanoCategory=Non-Violent Crime, Evaluation Protocol=Definition2026.05 | 19.2 | |
| GPT-5.4 MiniCategory=Harassment, Evaluation Protocol=Definition2026.05 | 22.9 | |
| GPT-5.4Category=Harassment, Evaluation Protocol=Definition2026.05 | 24.1 | |
| GPT-5.4 NanoCategory=Harassment, Evaluation Protocol=Definition2026.05 | 40.4 |