Malicious Code Generation on Malicious Code Generation
97Refusal Rate (Overall)Gemma-2B-IT
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Gemma-2B-ITSuffix Type=No Suffix2026.06 | 97 | — | — | — | — | — | — | |
| Phi-3-mini-128k-InstructSuffix Type=No Suffix2026.06 | 96 | — | — | — | — | — | — | |
| Llama-3.2-3B-InstructSuffix Type=No Suffix2026.06 | 86 | — | — | — | — | — | — | |
| Llama-3.1-8B-InstructSuffix Type=No Suffix2026.06 | 78 | — | — | — | — | — | — | |
| Phi-3-mini-128k-InstructSuffix Type=Super Suffix2026.06 | 44 | — | — | — | — | — | 93 | |
| Llama-3.2-3B-InstructSuffix Type=Super Suffix2026.06 | 42 | — | — | — | — | — | 93 | |
| Llama-3.1-8B-InstructSuffix Type=Super Suffix2026.06 | 38 | — | — | — | — | — | 96 | |
| Gemma-2B-ITSuffix Type=Super Suffix2026.06 | 35 | — | — | — | — | — | 94 | |
| Vicuna-7B-v1.5Suffix Type=No Suffix2026.06 | 35 | — | — | — | — | — | — | |
| Phi-3-mini-128k-InstructSuffix Type=Primary Suffix2026.06 | 21 | — | — | — | — | — | 29 | |
| Llama-3.2-3B-InstructSuffix Type=Primary Suffix2026.06 | 19 | — | — | — | — | — | 42 | |
| Gemma-2B-ITSuffix Type=Primary Suffix2026.06 | 13 | — | — | — | — | — | 43 | |
| Llama-3.1-8B-InstructSuffix Type=Primary Suffix2026.06 | 12 | — | — | — | — | — | 52 | |
| Vicuna-7B-v1.5Suffix Type=Super Suffix2026.06 | 5 | — | — | — | — | — | 93 | |
| Vicuna-7B-v1.5Suffix Type=Primary Suffix2026.06 | 0 | — | — | — | — | — | 45 | |
| google/gemma-2b-itModel Family=Gemma, Parameters=2B2025.12 | — | 97 | 13 | 43 | 35 | 94 | — | |
| lmsys/vicuna-7b-v1.5Model Family=Vicuna, Parameters=7B2025.12 | — | 35 | 0 | 45 | 5 | 93 | — | |
| meta-llama/Llama-3.1-8B-instructModel Family=Llama, Parameters=8B2025.12 | — | 78 | 12 | 52 | 38 | 96 | — | |
| meta-llama/Llama-3.2-3B-instructModel Family=Llama, Parameters=3B2025.12 | — | 86 | 19 | 42 | 42 | 93 | — | |
| microsoft/Phi-3-mini-128k-instructModel Family=Phi, Context Window=128k2025.12 | — | 96 | 21 | 29 | 44 | 93 | — |