Safety Evaluation on HarmBench (Safety Score %)
0ASRSFT
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| SFTModel=DeepSeek-8B2026.05 | 0 | — | — | — | |
| SFTModel=Mistral-7B2026.05 | 0 | — | — | — | |
| SFTModel=Vicuna-7B2026.05 | 0 | — | — | — | |
| AdvGRPO (ST)Attacker=Qwen2.5-7B2026.06 | 0.9 | — | — | — | |
| LCDDModel=Mistral-7B, Sparsity=59.34%2026.05 | 1 | — | — | — | |
| SFTModel=Qwen3-0.6B2026.05 | 1.5 | — | — | — | |
| LCDDModel=DeepSeek-8B, Sparsity=81.38%2026.05 | 1.5 | — | — | — | |
| AdvGRPO (ST)Attacker=Qwen2.5-14B2026.06 | 1.5 | — | — | — | |
| LCDDModel=Vicuna-7B, Sparsity=84.40%2026.05 | 2 | — | — | — | |
| LCDDModel=Qwen3-0.6B, Sparsity=81.05%2026.05 | 2.5 | — | — | — | |
| InstructModel=Llama3, Fine-tuning Source=Alpaca, Selection Protocol=Top-1000 safety-degrading samples2026.05 | 3.5 | — | — | — | |
| InstructModel=Llama3, Fine-tuning Source=Dolly, Selection Protocol=Top-1000 safety-degrading samples2026.05 | 3.5 | — | — | — | |
| SETModel=Qwen2.5-14B-base, Fine-tuning Dataset=CB-Safety2026.04 | 4.1 | — | — | — | |
| AdvGameAttacker=Qwen2.5-7B2026.06 | 4.7 | — | — | — | |
| Moderate-GiModel=LLaMA-3.1-8B2026.04 | 5 | — | — | — | |
| SETModel=Llama3-8B-base, Fine-tuning Dataset=CB-Safety2026.04 | 5.2 | — | — | — | |
| InstructModel=Qwen2.5, Fine-tuning Source=Alpaca, Selection Protocol=Top-1000 safety-degrading samples2026.05 | 6.5 | — | — | — | |
| InstructModel=Qwen2.5, Fine-tuning Source=Dolly, Selection Protocol=Top-1000 safety-degrading samples2026.05 | 6.5 | — | — | — | |
| Qwen3.5-9BParameters=9B, Variant=Thinking2026.05 | 6.6 | — | — | — | |
| SETModel=Qwen2.5-7B-base, Fine-tuning Dataset=CB-Safety2026.04 | 7.2 | — | — | — | |
| SETModel=Qwen2.5-14B-base, Fine-tuning Dataset=R1-Safety2026.04 | 7.4 | — | — | — | |
| SETModel=Llama3-8B-base, Fine-tuning Dataset=R1-Safety2026.04 | 7.4 | — | — | — | |
| GradSafeModel=Llama3.1, Fine-tuning Source=Alpaca, Selection Protocol=Top-1000 safety-degrading samples2026.05 | 7.5 | — | — | — | |
| RandomModel=Qwen2.5, Fine-tuning Source=Alpaca, Selection Protocol=Top-1000 safety-degrading samples2026.05 | 9 | — | — | — | |
| SEALModel=Qwen2.5, Fine-tuning Source=Alpaca, Selection Protocol=Top-1000 safety-degrading samples2026.05 | 9 | — | — | — | |
| InstructModel=Llama3.1, Fine-tuning Source=Alpaca, Selection Protocol=Top-1000 safety-degrading samples2026.05 | 9.5 | — | — | — | |
| InstructModel=Llama3.1, Fine-tuning Source=Dolly, Selection Protocol=Top-1000 safety-degrading samples2026.05 | 9.5 | — | — | — | |
| O-LoRAModel=LLaMA-3.1-8B2026.04 | 10 | — | — | — | |
| RandomModel=Qwen2.5, Fine-tuning Source=Dolly, Selection Protocol=Top-1000 safety-degrading samples2026.05 | 10.5 | — | — | — | |
| Mellum 2 (SFT)Post-training Stage=SFT, Parameters=2.5B/12B, Variant=Thinking2026.05 | 12.2 | — | — | — | |
| TrigModel=DeepSeek-8B, Sparsity=81.38%2026.05 | 13 | — | — | — | |
| SEALModel=Llama3, Fine-tuning Source=Dolly, Selection Protocol=Top-1000 safety-degrading samples2026.05 | 13 | — | — | — | |
| SEALModel=Llama3.1, Fine-tuning Source=Alpaca, Selection Protocol=Top-1000 safety-degrading samples2026.05 | 13 | — | — | — | |
| RandomModel=Llama3.1, Fine-tuning Source=Dolly, Selection Protocol=Top-1000 safety-degrading samples2026.05 | 13 | — | — | — | |
| GradSafeModel=Qwen2.5, Fine-tuning Source=Alpaca, Selection Protocol=Top-1000 safety-degrading samples2026.05 | 13 | — | — | — | |
| SEALModel=Llama3, Fine-tuning Source=Alpaca, Selection Protocol=Top-1000 safety-degrading samples2026.05 | 13.5 | — | — | — | |
| SafeLoRAModel=Llama3-8B-base, Fine-tuning Dataset=CB-Safety2026.04 | 13.6 | — | — | — | |
| RandomModel=Llama3.1, Fine-tuning Source=Alpaca, Selection Protocol=Top-1000 safety-degrading samples2026.05 | 14 | — | — | — | |
| Qwen3.5-4BParameters=4B, Variant=Thinking2026.05 | 15.9 | — | — | — | |
| Bi-AnchorModel=Llama3.1, Fine-tuning Source=Alpaca, Selection Protocol=Top-1000 safety-degrading samples2026.05 | 16.5 | — | — | — | |
| Bi-AnchorModel=Qwen2.5, Fine-tuning Source=Alpaca, Selection Protocol=Top-1000 safety-degrading samples2026.05 | 16.5 | — | — | — | |
| Self-RedTeamAttacker=Qwen2.5-7B2026.06 | 16.8 | — | — | — | |
| SN-tuneModel=Llama3-8B-base, Fine-tuning Dataset=CB-Safety2026.04 | 17 | — | — | — | |
| TrigModel=Mistral-7B, Sparsity=59.34%2026.05 | 17 | — | — | — | |
| RandomModel=LLaMA-3.1-8B2026.04 | 17.2 | — | — | — | |
| SafeLoRAModel=Qwen2.5-14B-base, Fine-tuning Dataset=CB-Safety2026.04 | 17.9 | — | — | — | |
| LoRAModel=Llama3-8B-base, Fine-tuning Dataset=CB-Safety2026.04 | 18.4 | — | — | — | |
| SVGTBackbone=Llama-3.2-3B, Prefix Length (K)=10, Evaluation Protocol=Identical as Table 22026.05 | 18.5 | — | — | 75.5 | |
| RandomModel=Llama3, Fine-tuning Source=Alpaca, Selection Protocol=Top-1000 safety-degrading samples2026.05 | 18.5 | — | — | — | |
| BaseAttacker=None2026.06 | 18.8 | — | — | — | |
| BaseModel=DeepSeek-8B2026.05 | 19 | — | — | — | |
| GuardModel=Llama3.1, Fine-tuning Source=Alpaca, Selection Protocol=Top-1000 safety-degrading samples2026.05 | 19.5 | — | — | — | |
| RandomModel=Llama3, Fine-tuning Source=Dolly, Selection Protocol=Top-1000 safety-degrading samples2026.05 | 20 | — | — | — | |
| SETModel=Qwen2.5-7B-base, Fine-tuning Dataset=R1-Safety2026.04 | 20.3 | — | — | — | |
| BaseModel=Vicuna-7B2026.05 | 20.5 | — | — | — | |
| Mellum 2 (RL)Post-training Stage=RL, Parameters=2.5B/12B, Variant=Thinking2026.05 | 20.6 | — | — | — | |
| GradSafeModel=Llama3, Fine-tuning Source=Alpaca, Selection Protocol=Top-1000 safety-degrading samples2026.05 | 21 | — | — | — | |
| SN-tuneModel=Qwen2.5-14B-base, Fine-tuning Dataset=CB-Safety2026.04 | 21.8 | — | — | — | |
| TrigModel=Vicuna-7B, Sparsity=84.40%2026.05 | 22 | — | — | — | |
| GuardModel=Qwen2.5, Fine-tuning Source=Alpaca, Selection Protocol=Top-1000 safety-degrading samples2026.05 | 22 | — | — | — | |
| SafeLoRAModel=Llama3-8B-base, Fine-tuning Dataset=R1-Safety2026.04 | 22.1 | — | — | — | |
| LoRAModel=Qwen2.5-14B-base, Fine-tuning Dataset=CB-Safety2026.04 | 23.4 | — | — | — | |
| QwQConflict=direct_q2026.04 | 23.5 | — | — | — | |
| DataShieldModel=Qwen2.5, Fine-tuning Source=Alpaca, Selection Protocol=Top-1000 safety-degrading samples2026.05 | 23.5 | — | — | — | |
| LARFModel=Qwen2.5, Fine-tuning Source=Alpaca, Selection Protocol=Top-1000 safety-degrading samples2026.05 | 24.5 | — | — | — | |
| GuardModel=Llama3, Fine-tuning Source=Alpaca, Selection Protocol=Top-1000 safety-degrading samples2026.05 | 25 | — | — | — | |
| SafeLoRAModel=Qwen2.5-7B-base, Fine-tuning Dataset=CB-Safety2026.04 | 25.4 | — | — | — | |
| SN-tuneModel=Llama3-8B-base, Fine-tuning Dataset=R1-Safety2026.04 | 25.9 | — | — | — | |
| LoRAModel=Llama3-8B-base, Fine-tuning Dataset=R1-Safety2026.04 | 26.9 | — | — | — | |
| KLModel=LLaMA-3.1-8B2026.04 | 27.7 | — | — | — | |
| BaselineModel=LLaMA-3.1-8B2026.04 | 27.8 | — | — | — | |
| SafeLoRAModel=Qwen2.5-14B-base, Fine-tuning Dataset=R1-Safety2026.04 | 28.9 | — | — | — | |
| DExpertsBackbone=Llama-3.2-3B, Control Category=Decoding-time controlled generation, Evaluation Protocol=Identical as Table 22026.05 | 29 | — | — | 66.5 | |
| Bi-AnchorModel=Llama3, Fine-tuning Source=Alpaca, Selection Protocol=Top-1000 safety-degrading samples2026.05 | 29.5 | — | — | — | |
| SN-tuneModel=Qwen2.5-7B-base, Fine-tuning Dataset=CB-Safety2026.04 | 29.7 | — | — | — | |
| LARFModel=Llama3.1, Fine-tuning Source=Alpaca, Selection Protocol=Top-1000 safety-degrading samples2026.05 | 30 | — | — | — | |
| EWCModel=LLaMA-3.1-8B2026.04 | 31 | — | — | — | |
| Prefix TuningBackbone=Llama-3.2-3B, Control Category=Prefix-style attention targets, Prefix Length (K)=10, Evaluation Protocol=Identical as Table 22026.05 | 31.5 | — | — | 68 | |
| LoRAModel=Qwen2.5-7B-base, Fine-tuning Dataset=CB-Safety2026.04 | 31.8 | — | — | — | |
| GeDiBackbone=Llama-3.2-3B, Control Category=Decoding-time controlled generation, Evaluation Protocol=Identical as Table 22026.05 | 32 | — | — | 56 | |
| LARFModel=Llama3, Fine-tuning Source=Alpaca, Selection Protocol=Top-1000 safety-degrading samples2026.05 | 32 | — | — | — | |
| DataShieldModel=Llama3.1, Fine-tuning Source=Alpaca, Selection Protocol=Top-1000 safety-degrading samples2026.05 | 32 | — | — | — | |
| RandomModel=Llama3-8B-base, Fine-tuning Dataset=CB-Safety2026.04 | 32.7 | — | — | — | |
| SN-tuneModel=Qwen2.5-14B-base, Fine-tuning Dataset=R1-Safety2026.04 | 33.2 | — | — | — | |
| GradSafeModel=Qwen2.5, Fine-tuning Source=Dolly, Selection Protocol=Top-1000 safety-degrading samples2026.05 | 33.5 | — | — | — | |
| LoRAModel=Qwen2.5-14B-base, Fine-tuning Dataset=R1-Safety2026.04 | 34.6 | — | — | — | |
| RandomModel=Llama3-8B-base, Fine-tuning Dataset=R1-Safety2026.04 | 34.8 | — | — | — | |
| QwQConflict=dilemma2026.04 | 36.5 | — | 13 | — | |
| TrigModel=Qwen3-0.6B, Sparsity=81.05%2026.05 | 36.5 | — | — | — | |
| GuardModel=Llama3.1, Fine-tuning Source=Dolly, Selection Protocol=Top-1000 safety-degrading samples2026.05 | 38 | — | — | — | |
| SafeLoRAModel=Qwen2.5-7B-base, Fine-tuning Dataset=R1-Safety2026.04 | 39.2 | — | — | — | |
| BaseModel=Llama3-8B-base, Fine-tuning Dataset=R1-Safety2026.04 | 41.2 | — | — | — | |
| BaseModel=Llama3-8B-base, Fine-tuning Dataset=CB-Safety2026.04 | 41.2 | — | — | — | |
| QwQConflict=inner2026.04 | 42 | — | 18.5 | — | |
| BaseModel=Qwen3-0.6B2026.05 | 43 | — | — | — | |
| SN-tuneModel=Qwen2.5-7B-base, Fine-tuning Dataset=R1-Safety2026.04 | 43.7 | — | — | — | |
| LoRAModel=Qwen2.5-7B-base, Fine-tuning Dataset=R1-Safety2026.04 | 44.9 | — | — | — | |
| RandomModel=Qwen2.5-14B-base, Fine-tuning Dataset=CB-Safety2026.04 | 46.2 | — | — | — | |
| RandomModel=Qwen2.5-14B-base, Fine-tuning Dataset=R1-Safety2026.04 | 47.3 | — | — | — | |
| PPLMBackbone=Llama-3.2-3B, Control Category=Decoding-time controlled generation, Evaluation Protocol=Identical as Table 22026.05 | 48.5 | — | — | 47 |