Over-refusal Evaluation Suite (XSTest, WildJailbreak, WildGuard, OKTest, OR-Bench)
3.2XSTest Refusal Rate (%)Pure
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| PureModel Size=Qwen3-4B2026.02 | 3.2 | 0 | 1.9 | 5.33 | 19.79 | 6.04 | |
| PureModel Size=Qwen3-8B2026.02 | 3.2 | 0.48 | 1.27 | 5.33 | 15.09 | 5.07 | |
| PureModel Size=Qwen3-14B2026.02 | 3.6 | 1.43 | 1.48 | 7.33 | 15.62 | 5.89 | |
| ShortModel Size=Qwen3-4B2026.02 | 4.4 | 1.43 | 2.22 | 12.67 | 41.93 | 12.53 | |
| SafeChainModel Size=Qwen3-14B2026.02 | 4.4 | 2.38 | 3.92 | 9.33 | 10.61 | 6.13 | |
| ShortModel Size=Qwen3-8B2026.02 | 4.8 | 3.33 | 4.23 | 10.33 | 40.86 | 12.71 | |
| ShortModel Size=Qwen3-14B2026.02 | 5.2 | 3.81 | 4.02 | 12.33 | 46.85 | 14.44 | |
| SafeChainModel Size=Qwen3-8B2026.02 | 6.4 | 0 | 3.17 | 7.67 | 9.25 | 5.3 | |
| IFPOModel Size=Qwen3-14B2026.02 | 7.2 | 1.43 | 3.6 | 9.67 | 7.28 | 5.84 | |
| IFPOModel Size=Qwen3-4B2026.02 | 7.6 | 2.38 | 4.34 | 9.67 | 5.99 | 6 | |
| IFPOModel Size=Qwen3-8B2026.02 | 7.6 | 2.38 | 4.34 | 9.67 | 5.99 | 4.33 | |
| STAR-1-mixModel Size=Qwen3-14B2026.02 | 8 | 1.9 | 4.13 | 12.67 | 48.29 | 15 | |
| SafeChainModel Size=Qwen3-4B2026.02 | 8.8 | 5.24 | 4.97 | 9.33 | 12.81 | 8.23 | |
| AgentModel Size=Qwen3-4B2026.02 | 10.4 | 6.67 | 6.88 | 14 | 79.15 | 23.42 | |
| STAR-1-mixModel Size=Qwen3-8B2026.02 | 10.8 | 0.48 | 4.23 | 10.33 | 50.34 | 15.24 | |
| BCModel Size=Qwen3-14B2026.02 | 11.2 | 4.76 | 5.71 | 14.67 | 48.6 | 16.99 | |
| AgentModel Size=Qwen3-8B2026.02 | 11.6 | 7.62 | 7.3 | 15.33 | 71.72 | 22.71 | |
| BCModel Size=Qwen3-8B2026.02 | 12.4 | 1.9 | 4.44 | 12.33 | 43.82 | 14.98 | |
| STAR-1Model Size=Qwen3-14B2026.02 | 12.8 | 2.86 | 6.14 | 17 | 55.12 | 18.78 | |
| BCModel Size=Qwen3-4B2026.02 | 13.6 | 3.81 | 4.66 | 12.33 | 47.23 | 16.33 | |
| STAR-1Model Size=Qwen3-8B2026.02 | 14 | 3.81 | 6.67 | 15.67 | 54.51 | 18.93 | |
| AgentModel Size=Qwen3-14B2026.02 | 14 | 19.05 | 11.75 | 16.67 | 80.59 | 28.41 | |
| STAR-1-mixModel Size=Qwen3-4B2026.02 | 16 | 1.43 | 4.97 | 11.33 | 57.16 | 18.18 | |
| STAR-1Model Size=Qwen3-4B2026.02 | 19.2 | 1.9 | 7.41 | 18.33 | 63.31 | 22.03 |