Malicious Prompt Refusal on HarmBench
96Refusal Ratemicrosoft/Phi-3-mini-128k-instruct
Evaluation results
| Method | Links | ||
|---|---|---|---|
| microsoft/Phi-3-mini-128k-instructSuffix Type=No Suffix2025.12 | 96 | — | |
| Phi-3-mini-128k-InstructSuffix Type=No Suffix2026.06 | 96 | — | |
| google/gemma-2b-itSuffix Type=No Suffix2025.12 | 94 | — | |
| Gemma-2B-ITSuffix Type=No Suffix2026.06 | 94 | — | |
| meta-llama/Llama-3.1-8B-instructSuffix Type=No Suffix2025.12 | 89 | — | |
| Llama-3.1-8B-InstructSuffix Type=No Suffix2026.06 | 89 | — | |
| meta-llama/Llama-3.2-3B-instructSuffix Type=No Suffix2025.12 | 84 | — | |
| Llama-3.2-3B-InstructSuffix Type=No Suffix2026.06 | 84 | — | |
| SafeAnchorBase Model=Llama-2-7B-Chat, Attack Method=GCG, Adaptation Pipeline=default-ordering T=32026.04 | 78.4 | — | |
| lmsys/vicuna-7b-v1.5Suffix Type=Primary Suffix2025.12 | 76 | 0.32 | |
| Vicuna-7B-v1.5Suffix Type=Primary Suffix2026.06 | 76 | 32 | |
| google/gemma-2b-itSuffix Type=Super Suffix2025.12 | 62 | 0.91 | |
| Gemma-2B-ITSuffix Type=Super Suffix2026.06 | 62 | 91 | |
| microsoft/Phi-3-mini-128k-instructSuffix Type=Super Suffix2025.12 | 59 | 0.86 | |
| Phi-3-mini-128k-InstructSuffix Type=Super Suffix2026.06 | 59 | 86 | |
| lmsys/vicuna-7b-v1.5Suffix Type=Super Suffix2025.12 | 58 | 0.86 | |
| Vicuna-7B-v1.5Suffix Type=Super Suffix2026.06 | 58 | 86 | |
| meta-llama/Llama-3.1-8B-instructSuffix Type=Super Suffix2025.12 | 55 | 0.88 | |
| meta-llama/Llama-3.2-3B-instructSuffix Type=Super Suffix2025.12 | 55 | 0.9 | |
| Llama-3.1-8B-InstructSuffix Type=Super Suffix2026.06 | 55 | 88 | |
| Llama-3.2-3B-InstructSuffix Type=Super Suffix2026.06 | 55 | 90 | |
| SafeGrad + LoRABase Model=Llama-2-7B-Chat, Attack Method=GCG, Adaptation Pipeline=default-ordering T=32026.04 | 54.6 | — | |
| meta-llama/Llama-3.2-3B-instructSuffix Type=Primary Suffix2025.12 | 52 | 0.2 | |
| Llama-3.2-3B-InstructSuffix Type=Primary Suffix2026.06 | 52 | 20 | |
| google/gemma-2b-itSuffix Type=Primary Suffix2025.12 | 51 | 0.35 | |
| lmsys/vicuna-7b-v1.5Suffix Type=No Suffix2025.12 | 51 | — | |
| microsoft/Phi-3-mini-128k-instructSuffix Type=Primary Suffix2025.12 | 51 | 0.17 | |
| Gemma-2B-ITSuffix Type=Primary Suffix2026.06 | 51 | 35 | |
| Vicuna-7B-v1.5Suffix Type=No Suffix2026.06 | 51 | — | |
| Phi-3-mini-128k-InstructSuffix Type=Primary Suffix2026.06 | 51 | 17 | |
| Safety InterleavingBase Model=Llama-2-7B-Chat, Attack Method=GCG, Adaptation Pipeline=default-ordering T=32026.04 | 49.3 | — | |
| Vaccine + LoRABase Model=Llama-2-7B-Chat, Attack Method=GCG, Adaptation Pipeline=default-ordering T=32026.04 | 46.5 | — | |
| Safe LoRABase Model=Llama-2-7B-Chat, Attack Method=GCG, Adaptation Pipeline=default-ordering T=32026.04 | 44.7 | — | |
| EWC + LoRABase Model=Llama-2-7B-Chat, Attack Method=GCG, Adaptation Pipeline=default-ordering T=32026.04 | 36.4 | — | |
| O-LoRABase Model=Llama-2-7B-Chat, Attack Method=GCG, Adaptation Pipeline=default-ordering T=32026.04 | 34.8 | — | |
| Standard LoRABase Model=Llama-2-7B-Chat, Attack Method=GCG, Adaptation Pipeline=default-ordering T=32026.04 | 31.2 | — | |
| meta-llama/Llama-3.1-8B-instructSuffix Type=Primary Suffix2025.12 | 20 | 0.23 | |
| Llama-3.1-8B-InstructSuffix Type=Primary Suffix2026.06 | 20 | 23 |