Harmful Prompt Refusal on HarmBench
0ASRRDO
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| RDOTarget Model=GPT-OSS-20B, Reasoning (R)=true, Multimodal (MM)=false2026.05 | 0 | — | — | — | — | — | — | — | — | — | — | — | |
| PSTarget Model=GPT-OSS-20B, Reasoning (R)=true, Multimodal (MM)=false2026.05 | 0 | — | — | — | — | — | — | — | — | — | — | — | |
| PSTarget Model=DeepSeek-R1-8B, Reasoning (R)=true, Multimodal (MM)=false2026.05 | 0 | — | — | — | — | — | — | — | — | — | — | — | |
| DiMTarget Model=GPT-OSS-20B, Reasoning (R)=true, Multimodal (MM)=false2026.05 | 0.6 | — | — | — | — | — | — | — | — | — | — | — | |
| DiMTarget Model=Phi-4-15B, Reasoning (R)=true, Multimodal (MM)=true2026.05 | 0.6 | — | — | — | — | — | — | — | — | — | — | — | |
| RDOTarget Model=Phi-4-15B, Reasoning (R)=true, Multimodal (MM)=true2026.05 | 0.6 | — | — | — | — | — | — | — | — | — | — | — | |
| PSTarget Model=Phi-4-15B, Reasoning (R)=true, Multimodal (MM)=true2026.05 | 0.6 | — | — | — | — | — | — | — | — | — | — | — | |
| RDOTarget Model=Qwen3.5-9B, Reasoning (R)=true, Multimodal (MM)=true2026.05 | 1.25 | — | — | — | — | — | — | — | — | — | — | — | |
| DiMTarget Model=Qwen3.5-9B, Reasoning (R)=true, Multimodal (MM)=true2026.05 | 3.31 | — | — | — | — | — | — | — | — | — | — | — | |
| PSTarget Model=Qwen3.5-9B, Reasoning (R)=true, Multimodal (MM)=true2026.05 | 3.77 | — | — | — | — | — | — | — | — | — | — | — | |
| PSTarget Model=Gemma3-12B, Reasoning (R)=false, Multimodal (MM)=true2026.05 | 7.54 | — | — | — | — | — | — | — | — | — | — | — | |
| MDTarget Model=Qwen3.5-9B, Reasoning (R)=true, Multimodal (MM)=true2026.05 | 10.06 | — | — | — | — | — | — | — | — | — | — | — | |
| RDOTarget Model=Ministral3-14B, Reasoning (R)=false, Multimodal (MM)=true2026.05 | 10.06 | — | — | — | — | — | — | — | — | — | — | — | |
| RDOTarget Model=Gemma3-12B, Reasoning (R)=false, Multimodal (MM)=true2026.05 | 14.46 | — | — | — | — | — | — | — | — | — | — | — | |
| Self-PlayBase Model=Llama-3.1-8B-IT, Input Condition=adv harm2025.10 | 20.7 | — | — | — | — | — | — | — | — | — | — | — | |
| Self-Play + SFTBase Model=Llama-3.1-8B-IT, Input Condition=adv harm2025.10 | 22.1 | — | — | — | — | — | — | — | — | — | — | — | |
| Defender-OnlyBase Model=Llama-3.1-8B-IT, Input Condition=adv harm2025.10 | 24.3 | — | — | — | — | — | — | — | — | — | — | — | |
| Defender-Only + SFTBase Model=Llama-3.1-8B-IT, Input Condition=adv harm2025.10 | 26 | — | — | — | — | — | — | — | — | — | — | — | |
| ELSBase Model=Llama-3.1-8B-IT, Input Condition=adv harm2025.10 | 28.9 | — | — | — | — | — | — | — | — | — | — | — | |
| ABSBackbone=Qwen2.5-3B-IT2026.05 | 32 | — | — | — | — | — | — | — | — | — | — | — | |
| SFTBase Model=Llama-3.1-8B-IT, Input Condition=adv harm2025.10 | 34.8 | — | — | — | — | — | — | — | — | — | — | — | |
| ABSBackbone=Qwen2.5-7B-IT2026.05 | 38.3 | — | — | — | — | — | — | — | — | — | — | — | |
| RDOTarget Model=DeepSeek-R1-8B, Reasoning (R)=true, Multimodal (MM)=false2026.05 | 40.25 | — | — | — | — | — | — | — | — | — | — | — | |
| SELF-REDTEAMBackbone=Qwen2.5-7B-IT2026.05 | 42.3 | — | — | — | — | — | — | — | — | — | — | — | |
| ABSBackbone=Qwen2.5-14B-IT2026.05 | 44.6 | — | — | — | — | — | — | — | — | — | — | — | |
| SELF-REDTEAMBackbone=Qwen2.5-14B-IT2026.05 | 46.7 | — | — | — | — | — | — | — | — | — | — | — | |
| PSTarget Model=Ministral3-14B, Reasoning (R)=false, Multimodal (MM)=true2026.05 | 50.31 | — | — | — | — | — | — | — | — | — | — | — | |
| MDTarget Model=GPT-OSS-20B, Reasoning (R)=true, Multimodal (MM)=false2026.05 | 52.2 | — | — | — | — | — | — | — | — | — | — | — | |
| MDTarget Model=Gemma3-12B, Reasoning (R)=false, Multimodal (MM)=true2026.05 | 53.45 | — | — | — | — | — | — | — | — | — | — | — | |
| DiMTarget Model=DeepSeek-R1-8B, Reasoning (R)=true, Multimodal (MM)=false2026.05 | 57.23 | — | — | — | — | — | — | — | — | — | — | — | |
| MDTarget Model=Phi-4-15B, Reasoning (R)=true, Multimodal (MM)=true2026.05 | 57.23 | — | — | — | — | — | — | — | — | — | — | — | |
| CLE-PTarget Model=Phi-4-15B, Reasoning (R)=true, Multimodal (MM)=true2026.05 | 59.74 | — | — | — | — | — | — | — | — | — | — | — | |
| SELF-REDTEAMBackbone=Qwen2.5-3B-IT2026.05 | 60.3 | — | — | — | — | — | — | — | — | — | — | — | |
| Qwen2.5-7B-ITBackbone=Qwen2.5-7B-IT2026.05 | 61.3 | — | — | — | — | — | — | — | — | — | — | — | |
| Qwen2.5-14B-ITBackbone=Qwen2.5-14B-IT2026.05 | 64.7 | — | — | — | — | — | — | — | — | — | — | — | |
| Llama-3.1-8B-ITModel Status=Base Model, Input Condition=adv harm2025.10 | 65.4 | — | — | — | — | — | — | — | — | — | — | — | |
| DiMTarget Model=Gemma3-12B, Reasoning (R)=false, Multimodal (MM)=true2026.05 | 66.66 | — | — | — | — | — | — | — | — | — | — | — | |
| Qwen2.5-3B-ITBackbone=Qwen2.5-3B-IT2026.05 | 67.7 | — | — | — | — | — | — | — | — | — | — | — | |
| MDTarget Model=DeepSeek-R1-8B, Reasoning (R)=true, Multimodal (MM)=false2026.05 | 78.61 | — | — | — | — | — | — | — | — | — | — | — | |
| CLE-ATarget Model=Phi-4-15B, Reasoning (R)=true, Multimodal (MM)=true2026.05 | 78.61 | — | — | — | — | — | — | — | — | — | — | — | |
| CLE-PTarget Model=DeepSeek-R1-8B, Reasoning (R)=true, Multimodal (MM)=false2026.05 | 84.91 | — | — | — | — | — | — | — | — | — | — | — | |
| CLE-PTarget Model=GPT-OSS-20B, Reasoning (R)=true, Multimodal (MM)=false2026.05 | 86.79 | — | — | — | — | — | — | — | — | — | — | — | |
| CLE-ATarget Model=DeepSeek-R1-8B, Reasoning (R)=true, Multimodal (MM)=false2026.05 | 86.79 | — | — | — | — | — | — | — | — | — | — | — | |
| CLE-PTarget Model=Qwen3.5-9B, Reasoning (R)=true, Multimodal (MM)=true2026.05 | 92.45 | — | — | — | — | — | — | — | — | — | — | — | |
| MDTarget Model=Ministral3-14B, Reasoning (R)=false, Multimodal (MM)=true2026.05 | 94.96 | — | — | — | — | — | — | — | — | — | — | — | |
| CLE-ATarget Model=GPT-OSS-20B, Reasoning (R)=true, Multimodal (MM)=false2026.05 | 95.6 | — | — | — | — | — | — | — | — | — | — | — | |
| CLE-PTarget Model=Gemma3-12B, Reasoning (R)=false, Multimodal (MM)=true2026.05 | 95.6 | — | — | — | — | — | — | — | — | — | — | — | |
| DiMTarget Model=Ministral3-14B, Reasoning (R)=false, Multimodal (MM)=true2026.05 | 96.22 | — | — | — | — | — | — | — | — | — | — | — | |
| CLE-PTarget Model=Ministral3-14B, Reasoning (R)=false, Multimodal (MM)=true2026.05 | 96.85 | — | — | — | — | — | — | — | — | — | — | — | |
| CLE-ATarget Model=Qwen3.5-9B, Reasoning (R)=true, Multimodal (MM)=true2026.05 | 97.48 | — | — | — | — | — | — | — | — | — | — | — | |
| CLE-ATarget Model=Ministral3-14B, Reasoning (R)=false, Multimodal (MM)=true2026.05 | 99.37 | — | — | — | — | — | — | — | — | — | — | — | |
| CLE-ATarget Model=Gemma3-12B, Reasoning (R)=false, Multimodal (MM)=true2026.05 | 100 | — | — | — | — | — | — | — | — | — | — | — | |
| baselineModel=Llama-3.1-8B2026.05 | — | — | — | — | — | — | — | — | — | — | — | 37 | |
| baselineModel=Mistral-7B2026.05 | — | — | — | — | — | — | — | — | — | — | — | 35 | |
| baselineModel=Qwen2.5-7B2026.05 | — | — | — | — | — | — | — | — | — | — | — | 8 | |
| baselineModel=Qwen3-4B2026.05 | — | — | — | — | — | — | — | — | — | — | — | 8.5 | |
| baselineModel=Gemma-3-4B2026.05 | — | — | — | — | — | — | — | — | — | — | — | 14 | |
| DPO mix50/1000/augModel=Llama-3.1-8B2026.05 | — | — | — | — | — | — | — | — | — | — | — | 35.5 | |
| DPO mix50/1000/augModel=Mistral-7B2026.05 | — | — | — | — | — | — | — | — | — | — | — | 28 | |
| DPO mix50/1000/augModel=Qwen3-4B2026.05 | — | — | — | — | — | — | — | — | — | — | — | 11 | |
| DPO mix50/1000/augModel=Gemma-3-4B2026.05 | — | — | — | — | — | — | — | — | — | — | — | 18 | |
| DPO mix50/1000/cleanModel=Qwen2.5-7B2026.05 | — | — | — | — | — | — | — | — | — | — | — | 7.5 | |
| DPO mix50/2368/aug ext. pool⋄Model=Qwen2.5-7B2026.05 | — | — | — | — | — | — | — | — | — | — | — | 12.7 | |
| Gemma 3 1BModel size=1B2025.07 | — | 96 | 61 | 27 | 9 | 44 | 18 | 84 | 37 | 73 | 25 | — | |
| Llama 2 7B DAModel size=7B, Defense Variant=Do-Not-Answer (DA)2025.07 | — | 4 | 1 | 4 | 0 | 12 | 2 | 54 | 12 | 37 | 6 | — | |
| Llama 3 CBDefense Variant=Circuit Breakers (CB)2025.07 | — | 8 | 1 | 3 | 1 | 5 | 1 | 13 | 4 | 12 | 3 | — | |
| Llama 3.1 8BModel size=8B2025.07 | — | 96 | 71 | 52 | 21 | 36 | 12 | 68 | 37 | 65 | 32 | — | |
| Llama-Guard-3Model=Llama-3.1-8B2026.05 | — | — | — | — | — | — | — | — | — | — | — | 10 | |
| Llama-Guard-3Model=Mistral-7B2026.05 | — | — | — | — | — | — | — | — | — | — | — | 7 | |
| Self-ReminderModel=Llama-3.1-8B2026.05 | — | — | — | — | — | — | — | — | — | — | — | 11 | |
| SFT mix0/200/cleanModel=Qwen2.5-7B2026.05 | — | — | — | — | — | — | — | — | — | — | — | 9 | |
| SFT mix50/1000/augModel=Mistral-7B2026.05 | — | — | — | — | — | — | — | — | — | — | — | 30.5 | |
| SFT mix50/1000/augModel=Qwen3-4B2026.05 | — | — | — | — | — | — | — | — | — | — | — | 7.5 | |
| SFT mix50/1000/augModel=Gemma-3-4B2026.05 | — | — | — | — | — | — | — | — | — | — | — | 1.5 | |
| SFT mix50/200/augModel=Llama-3.1-8B2026.05 | — | — | — | — | — | — | — | — | — | — | — | 29.5 | |
| SmoothLLMModel=Llama-3.1-8B, K=10, q=10% swap2026.05 | — | — | — | — | — | — | — | — | — | — | — | 5.6 | |
| SmoothLLMModel=Mistral-7B, K=10, q=10% swap2026.05 | — | — | — | — | — | — | — | — | — | — | — | 20.2 | |
| SmoothLLMModel=Qwen2.5-7B, K=10, q=10% swap2026.05 | — | — | — | — | — | — | — | — | — | — | — | 5.4 | |
| SmoothLLMModel=Qwen3-4B, K=10, q=10% swap2026.05 | — | — | — | — | — | — | — | — | — | — | — | 4.5 | |
| SmoothLLMModel=Gemma-3-4B, K=10, q=10% swap2026.05 | — | — | — | — | — | — | — | — | — | — | — | 16.8 |