Safety Evaluation on HarmBench, StrongReject, WildJailbreak, XSTest Suite
68.44HarmBench ScoreInitial
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| InitialSize=0.6B2026.01 | 68.44 | 66.45 | 52.8 | 5.2 | 48.22 | |
| SafePathSize=0.6B2026.01 | 67.61 | 60.06 | 52.8 | 4.4 | 46.22 | |
| SafeKeySize=0.6B2026.01 | 60.96 | 48.88 | 52.75 | 18.4 | 45.25 | |
| SafeChainSize=0.6B2026.01 | 58.64 | 72.84 | 49.6 | 0 | 45.2 | |
| STAR-1Size=0.6B2026.01 | 56.64 | 38.02 | 50.6 | 22.4 | 41.92 | |
| SafePathSize=1.7B2026.01 | 54.15 | 36.42 | 49.3 | 1.2 | 35.27 | |
| InitialSize=1.7B2026.01 | 52.66 | 36.1 | 51.1 | 1.2 | 35.27 | |
| SafeChainSize=1.7B2026.01 | 47.34 | 57.51 | 43.85 | 1.6 | 37.58 | |
| SafeKeySize=1.7B2026.01 | 46.84 | 18.21 | 48.85 | 8.8 | 30.68 | |
| DirectRefusalSize=0.6B2026.01 | 43.85 | 11.82 | 36.3 | 83.6 | 43.89 | |
| SafeChainSize=4B2026.01 | 43.69 | 41.21 | 39.65 | 2 | 31.64 | |
| SafeChainSize=8B2026.01 | 41.2 | 38.95 | 36.42 | 1.2 | 29.44 | |
| THINKSAFESize=0.6B2026.01 | 40.37 | 33.87 | 37.95 | 6.4 | 29.65 | |
| DirectRefusalSize=1.7B2026.01 | 38.54 | 5.75 | 35.75 | 61.6 | 35.41 | |
| InitialSize=4B2026.01 | 38.21 | 8.31 | 43 | 0.8 | 22.58 | |
| SafePathSize=4B2026.01 | 37.71 | 7.35 | 42.45 | 1.6 | 22.28 | |
| STAR-1Size=1.7B2026.01 | 37.38 | 7.67 | 46.6 | 10.8 | 25.61 | |
| SafePathSize=8B2026.01 | 35.22 | 6.71 | 39.45 | 1.2 | 20.64 | |
| InitialSize=8B2026.01 | 35.05 | 4.47 | 38.35 | 0.4 | 19.57 | |
| STAR-1Size=4B2026.01 | 33.72 | 5.75 | 35.15 | 6.8 | 20.36 | |
| DirectRefusalSize=4B2026.01 | 33.06 | 3.19 | 36.2 | 32 | 29.8 | |
| SafeKeySize=4B2026.01 | 32.39 | 3.19 | 32.95 | 0.8 | 17.33 | |
| THINKSAFESize=1.7B2026.01 | 28.74 | 9.58 | 29.2 | 2 | 17.38 | |
| SafeKeySize=8B2026.01 | 26.91 | 4.79 | 28.8 | 8.8 | 17.33 | |
| DirectRefusalSize=8B2026.01 | 24.42 | 1.92 | 28.05 | 37.6 | 23 | |
| STAR-1Size=8B2026.01 | 24.42 | 1.28 | 29.25 | 6.8 | 15.44 | |
| THINKSAFESize=4B2026.01 | 9.63 | 0.32 | 7.45 | 2.8 | 5.05 | |
| THINKSAFESize=8B2026.01 | 9.14 | 0.32 | 7.35 | 1.2 | 4.5 |