Safety Evaluation on StrongReject
0Attack Success RateSTAR-1
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| STAR-1Model Architecture=DeepSeek-R1-Distill-Llama-8B2026.05 | 0 | — | — | |
| STAR-1Model Architecture=DeepSeek-R1-Distill-Qwen-14B2026.05 | 0.6 | — | — | |
| SInternalModel Architecture=DeepSeek-R1-Distill-Qwen-14B2026.05 | 0.6 | — | — | |
| DirectRefusalModel Size=8B2026.01 | 0.64 | — | — | |
| NoneModel=Gemma 2 2B2026.04 | 1.3 | — | — | |
| SInternalModel Architecture=DeepSeek-R1-Distill-Llama-8B2026.05 | 1.6 | — | — | |
| STAR-1Model Architecture=DeepSeek-R1-Distill-Qwen-7B2026.05 | 2.9 | — | — | |
| NoneModel=Llama 3.2 3B2026.04 | 3.5 | — | — | |
| STAR-1Model Size=8B2026.01 | 3.51 | — | — | |
| PreSafebase_model=DeepSeek-R1-Distill-Qwen-7B2026.03 | 5.4 | — | — | |
| TRIDENT-EDGEBackbone Model=META-LLAMA-3.1-8B2025.05 | 6 | 2 | — | |
| QwQConflict=direct_q2026.04 | 6 | — | — | |
| SInternalModel Architecture=DeepSeek-R1-Distill-Qwen-7B2026.05 | 7 | — | — | |
| WILDBREAKBackbone Model=META-LLAMA-3.1-8B2025.05 | 8 | 2.09 | — | |
| ATTAQBackbone Model=META-LLAMA-3.1-8B2025.05 | 10 | 1.99 | — | |
| SafeKeyModel Size=8B2026.01 | 11.82 | — | — | |
| WILDCHATBackbone Model=META-LLAMA-3.1-8B2025.05 | 12 | 2.13 | — | |
| AARTBackbone Model=META-LLAMA-3.1-8B2025.05 | 14 | 2.24 | — | |
| HH_RLHFBackbone Model=META-LLAMA-3.1-8B2025.05 | 15 | 2.23 | — | |
| SAFE_RLHFBackbone Model=META-LLAMA-3.1-8B2025.05 | 15 | 2.31 | — | |
| R2Dbase_model=DeepSeek-R1-Distill-Qwen-7B2026.03 | 16.3 | — | — | |
| UnalignedBackbone Model=META-LLAMA-3.1-8B2025.05 | 18 | 2.46 | — | |
| SafeChainModel Architecture=DeepSeek-R1-Distill-Qwen-14B2026.05 | 24.9 | — | — | |
| SafePathbase_model=DeepSeek-R1-Distill-Qwen-7B2026.03 | 25.6 | — | — | |
| THINKSAFEModel Size=8B2026.01 | 26.52 | — | — | |
| SafeChainModel Architecture=DeepSeek-R1-Distill-Llama-8B2026.05 | 28.8 | — | — | |
| QwQConflict=inner2026.04 | 30.5 | — | 0.245 | |
| Llama2-7BModel=Llama2-7B, Condition=Vanilla2026.01 | 32 | — | — | |
| DirectRefusalModel Size=7B2026.01 | 33.55 | — | — | |
| SafeKeyModel Size=7B2026.01 | 33.87 | — | — | |
| Llama-NemotronConflict=direct_q2026.04 | 39.6 | — | — | |
| BaseModel Architecture=DeepSeek-R1-Distill-Qwen-14B2026.05 | 41.2 | — | — | |
| THINKSAFEModel Size=7B2026.01 | 41.85 | — | — | |
| QwQConflict=dilemma2026.04 | 44 | — | 0.38 | |
| BaseModel Architecture=DeepSeek-R1-Distill-Llama-8B2026.05 | 44.7 | — | — | |
| SafeChainbase_model=DeepSeek-R1-Distill-Qwen-7B2026.03 | 45.2 | — | — | |
| SafeChainModel Size=8B2026.01 | 46.33 | — | — | |
| Llama-NemotronConflict=inner2026.04 | 46.9 | — | 0.073 | |
| STAR-1Model Size=7B2026.01 | 47.92 | — | — | |
| SafeChainModel Architecture=DeepSeek-R1-Distill-Qwen-7B2026.05 | 48.9 | — | — | |
| Llama-NemotronConflict=dilemma2026.04 | 49.8 | — | 0.102 | |
| BaseModel Architecture=DeepSeek-R1-Distill-Qwen-7B2026.05 | 50.2 | — | — | |
| SafePathModel Size=8B2026.01 | 51.44 | — | — | |
| UnSafeChainbase_model=DeepSeek-R1-Distill-Qwen-7B2026.03 | 51.8 | — | — | |
| InitialModel Size=8B2026.01 | 53.99 | — | — | |
| SafeChainModel Size=7B2026.01 | 54.63 | — | — | |
| InitialModel Size=7B2026.01 | 63.58 | — | — | |
| ShipsModel=Qwen2.5-7B2026.01 | 64 | — | — | |
| SafePathModel Size=7B2026.01 | 64.86 | — | — | |
| DS-7Bbase_model=DeepSeek-R1-Distill-Qwen-7B2026.03 | 66 | — | — | |
| SafeKeyModel Size=1.5B2026.01 | 73.16 | — | — | |
| THINKSAFEModel Size=1.5B2026.01 | 74.12 | — | — | |
| SafeChainModel Size=1.5B2026.01 | 76.68 | — | — | |
| STAR-1Model Size=1.5B2026.01 | 77 | — | — | |
| InitialModel Size=1.5B2026.01 | 82.11 | — | — | |
| POModel=Gemma 2 2B2026.04 | 82.4 | — | — | |
| SafePathModel Size=1.5B2026.01 | 82.43 | — | — | |
| DirectRefusalModel Size=1.5B2026.01 | 82.75 | — | — | |
| NTPModel=Gemma 2 2B2026.04 | 83.1 | — | — | |
| DIMModel=Gemma 2 2B2026.04 | 85 | — | — | |
| POModel=Llama 3.2 3B2026.04 | 86.6 | — | — | |
| DIMModel=Llama 3.2 3B2026.04 | 87.2 | — | — | |
| NTPModel=Llama 3.2 3B2026.04 | 88.2 | — | — | |
| Llama3.1-8BModel=Llama3.1-8B, Condition=Vanilla2026.01 | 160 | — | — | |
| Qwen2.5-7BModel=Qwen2.5-7B, Condition=Vanilla2026.01 | 767 | — | — | |
| ShipsModel=Mistral-7B2026.01 | 831 | — | — | |
| ShipsModel=Llama3.1-8B2026.01 | 1,661 | — | — | |
| Mistral-7BModel=Mistral-7B, Condition=Vanilla2026.01 | 2,971 | — | — | |
| ShipsModel=Llama2-7B2026.01 | 3,387 | — | — | |
| Fine-TuningModel=Llama2-7B2026.01 | 7,188 | — | — | |
| Fine-TuningModel=Mistral-7B2026.01 | 8,083 | — | — | |
| Fine-TuningModel=Qwen2.5-7B2026.01 | 8,243 | — | — | |
| Fine-TuningModel=Llama3.1-8B2026.01 | 8,466 | — | — | |
| GOSVModel=Qwen2.5-7B2026.01 | 9,042 | — | — | |
| GOSVModel=Llama2-7B2026.01 | 9,105 | — | — | |
| GOSVModel=Llama3.1-8B2026.01 | 9,744 | — | — | |
| GOSVModel=Mistral-7B2026.01 | 9,936 | — | — | |
| BaseBackbone=Llama-3.1-8B-Instruct2025.03 | — | 0.4054 | — | |
| BaseBackbone=Qwen-2-7B-Instruct2025.03 | — | 0.3808 | — | |
| CoTBackbone=Llama-3.1-8B-Instruct2025.03 | — | 0.379 | — | |
| CoTBackbone=Qwen-2-7B-Instruct2025.03 | — | 0.3792 | — | |
| DPOBackbone=Llama-3.1-8B-Instruct2025.03 | — | 0.5054 | — | |
| DPOBackbone=Qwen-2-7B-Instruct2025.03 | — | 0.5026 | — | |
| DR-IRLBackbone=Llama-3.1-8B-Instruct2025.03 | — | 0.9361 | — | |
| DR-IRLBackbone=Qwen-2-7B-Instruct2025.03 | — | 0.8798 | — | |
| GRPOBackbone=Llama-3.1-8B-Instruct2025.03 | — | 0.8105 | — | |
| GRPOBackbone=Qwen-2-7B-Instruct2025.03 | — | 0.5155 | — | |
| IRLBackbone=Llama-3.1-8B-Instruct2025.03 | — | 0.8917 | — | |
| IRLBackbone=Qwen-2-7B-Instruct2025.03 | — | 0.596 | — | |
| KLSelection Criterion=KL2026.04 | — | — | 31.9 | |
| Latent SimilaritySelection Criterion=Latent Similarity, Layer=122026.04 | — | — | 35.4 | |
| Latent SimilaritySelection Criterion=Latent Similarity, Layer=Best Layer2026.04 | — | — | 56.3 | |
| PerplexitySelection Criterion=Perplexity2026.04 | — | — | -8 | |
| RandomSelection Criterion=Random2026.04 | — | — | -2.6 | |
| SACPOBackbone=Llama-3.1-8B-Instruct2025.03 | — | 0.7264 | — | |
| SACPOBackbone=Qwen-2-7B-Instruct2025.03 | — | 0.5577 | — | |
| Self CertaintySelection Criterion=Self Certainty2026.04 | — | — | -87.2 | |
| Self-RewardingBackbone=Llama-3.1-8B-Instruct2025.03 | — | 0.4633 | — | |
| Self-RewardingBackbone=Qwen-2-7B-Instruct2025.03 | — | 0.5062 | — | |
| SFTBackbone=Llama-3.1-8B-Instruct2025.03 | — | 0.4698 | — |