Safety Over-triggering on ToxiGen
0.02Over-trigger Rate: JewishMistral
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mistral2026.05 | 0.02 | 0.03 | 0.06 | 0.04 | 0.02 | 0.04 | 0.04 | 0.01 | 0.05 | 0.06 | 0.02 | 0.03 | 0 | |
| Qwen2026.05 | 0.05 | 0.08 | 0.12 | 0.07 | 0.03 | 0.19 | 0.06 | 0.11 | 0.16 | 0.09 | 0.03 | 0.07 | 0.01 | |
| DeepSeek2026.05 | 0.06 | 0.02 | 0.04 | 0.06 | 0.01 | 0.04 | 0.04 | 0.06 | 0.08 | 0.05 | 0.03 | 0.09 | 0.03 | |
| Gemma2026.05 | 1.96 | 1.55 | 1.23 | 1.09 | 1.05 | 1.42 | 0.92 | 0.84 | 0.9 | 0.78 | 0.83 | 0.68 | 0.78 | |
| Airavata2026.05 | 9.74 | 9.56 | 7.35 | 6.58 | 7.44 | 7.72 | 7.19 | 6.64 | 6.65 | 5.83 | 6.47 | 4.76 | 3.63 | |
| Falcon2026.05 | 29.63 | 21.56 | 18.34 | 16.96 | 17.3 | 17.99 | 14.03 | 12.14 | 13.63 | 10.74 | 11.51 | 8.38 | 5.04 | |
| Llama2026.05 | 36.16 | 31.32 | 28.33 | 27.89 | 27.43 | 26.59 | 25.24 | 22.11 | 20.02 | 18.23 | 17.84 | 15.3 | 15.15 |