Jailbreak on Jailbreak scenario
95.7ASR (Jailbreak Scenario)LLM.int8()
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| LLM.int8()Model=Llama3.1 8B, Type=Zero-shot2026.05 | 95.7 | — | — | |
| NF4Model=Llama3.1 8B, Type=Zero-shot2026.05 | 95.7 | — | — | |
| GPTQ 4-bitModel=Llama3.1 8B, Type=DDOB2026.05 | 95.7 | — | — | |
| FP4Model=Llama3.1 8B, Type=Zero-shot2026.05 | 95 | — | — | |
| SINQModel=Qwen2.5 7B, Type=DIOB2026.05 | 95 | — | — | |
| AWQModel=Llama3.1 8B, Type=DDOB2026.05 | 95 | — | — | |
| NF4Model=Qwen2.5 7B, Type=Zero-shot2026.05 | 94.7 | — | — | |
| SINQModel=Llama3.1 8B, Type=DIOB2026.05 | 94.7 | — | — | |
| GPTQ 4-bitModel=Qwen2.5 7B, Type=DDOB2026.05 | 94.7 | — | — | |
| LLM.int8()Model=Qwen2.5 7B, Type=Zero-shot2026.05 | 94.3 | — | — | |
| HQQModel=Llama3.1 8B, Type=DIOB2026.05 | 94.3 | — | — | |
| AWQModel=Qwen2.5 7B, Type=DDOB2026.05 | 93.7 | — | — | |
| HQQModel=Qwen2.5 7B, Type=DIOB2026.05 | 93.3 | — | — | |
| FP4Model=Qwen2.5 7B, Type=Zero-shot2026.05 | 93 | — | — | |
| GPTQ 8-bitModel=Llama3.1 8B, Type=DDOB2026.05 | 92.7 | — | — | |
| GGUF Q4_0Model=Qwen2.5 7B, Type=Zero-shot2026.05 | 92 | — | — | |
| GGUF Q4_K_MModel=Qwen2.5 7B, Type=DIOB2026.05 | 91.3 | — | — | |
| GGUF IQ4_XSModel=Llama3.1 8B, Type=DDOB2026.05 | 91.3 | — | — | |
| GGUF Q4_K_MModel=Llama3.1 8B, Type=DIOB2026.05 | 91 | — | — | |
| HQQModel=Mistral 7B, Type=DIOB2026.05 | 90.7 | — | — | |
| NF4Model=Mistral 7B, Type=Zero-shot2026.05 | 90 | — | — | |
| GGUF Q4_0Model=Llama3.1 8B, Type=Zero-shot2026.05 | 90 | — | — | |
| GGUF Q4_K_MModel=Mistral 7B, Type=DIOB2026.05 | 89.7 | — | — | |
| GPTQ 4-bitModel=Mistral 7B, Type=DDOB2026.05 | 89.7 | — | — | |
| LLM.int8()Model=Mistral 7B, Type=Zero-shot2026.05 | 88 | — | — | |
| SINQModel=Mistral 7B, Type=DIOB2026.05 | 87.3 | — | — | |
| AWQModel=Mistral 7B, Type=DDOB2026.05 | 87.3 | — | — | |
| GPTQ 8-bitModel=Mistral 7B, Type=DDOB2026.05 | 87 | — | — | |
| AutoRoundModel=Mistral 7B, Type=DDOB2026.05 | 86.3 | — | — | |
| GPTQ 8-bitModel=Qwen2.5 7B, Type=DDOB2026.05 | 80.3 | — | — | |
| FP4Model=Mistral 7B, Type=Zero-shot2026.05 | 67.3 | — | — | |
| AutoRoundModel=Qwen2.5 7B, Type=DDOB2026.05 | 50 | — | — | |
| GGUF IQ4_XSModel=Mistral 7B, Type=DDOB2026.05 | 44.3 | — | — | |
| OriginalModel=Mistral 7B, Precision=BF162026.05 | 33.7 | — | — | |
| GGUF Q4_0Model=Mistral 7B, Type=Zero-shot2026.05 | 30.7 | — | — | |
| AttackedModel=Mistral 7B, Precision=BF162026.05 | 28.7 | — | — | |
| AutoRoundModel=Llama3.1 8B, Type=DDOB2026.05 | 21 | — | — | |
| GGUF IQ4_XSModel=Qwen2.5 7B, Type=DDOB2026.05 | 13.7 | — | — | |
| OriginalModel=Qwen2.5 7B, Precision=BF162026.05 | 12 | — | — | |
| OriginalModel=Llama3.1 8B, Precision=BF162026.05 | 8 | — | — | |
| AttackedModel=Qwen2.5 7B, Precision=BF162026.05 | 6.3 | — | — | |
| AttackedModel=Llama3.1 8B, Precision=BF162026.05 | 2.3 | — | — | |
| Llama3.1-8BModel=Llama3.1-8B, Pruning Method=Unpruned, Sparsity Ratio=-2025.10 | — | 2 | 7.3 | |
| Llama3.1-8BModel=Llama3.1-8B, Pruning Method=Mag., Sparsity Ratio=20%2025.10 | — | 92.3 | 9 | |
| Llama3.1-8BModel=Llama3.1-8B, Pruning Method=SparseGPT, Sparsity Ratio=20%2025.10 | — | 22 | 6 | |
| Llama3.1-8BModel=Llama3.1-8B, Pruning Method=SparseGPT, Sparsity Ratio=2:42025.10 | — | 19.3 | 18.7 | |
| Llama3.1-8BModel=Llama3.1-8B, Pruning Method=SparseGPT, Sparsity Ratio=50%2025.10 | — | 36 | 14.3 | |
| Llama3.1-8BModel=Llama3.1-8B, Pruning Method=Wanda, Sparsity Ratio=20%2025.10 | — | 93.3 | 6.7 | |
| Llama3.1-8BModel=Llama3.1-8B, Pruning Method=Wanda, Sparsity Ratio=2:42025.10 | — | 63.7 | 29.3 | |
| Llama3.1-8BModel=Llama3.1-8B, Pruning Method=Wanda, Sparsity Ratio=50%2025.10 | — | 92.3 | 16.3 | |
| OLMo-2-7BModel=OLMo-2-7B, Pruning Method=Unpruned, Sparsity Ratio=-2025.10 | — | 3 | 3 | |
| OLMo-2-7BModel=OLMo-2-7B, Pruning Method=Mag., Sparsity Ratio=20%2025.10 | — | 94.3 | 2.7 | |
| OLMo-2-7BModel=OLMo-2-7B, Pruning Method=SparseGPT, Sparsity Ratio=20%2025.10 | — | 92.7 | 3.3 | |
| OLMo-2-7BModel=OLMo-2-7B, Pruning Method=SparseGPT, Sparsity Ratio=2:42025.10 | — | 70.7 | 17 | |
| OLMo-2-7BModel=OLMo-2-7B, Pruning Method=SparseGPT, Sparsity Ratio=50%2025.10 | — | 89.3 | 5 | |
| OLMo-2-7BModel=OLMo-2-7B, Pruning Method=Wanda, Sparsity Ratio=20%2025.10 | — | 91.7 | 2.3 | |
| OLMo-2-7BModel=OLMo-2-7B, Pruning Method=Wanda, Sparsity Ratio=2:42025.10 | — | 75.3 | 21.7 | |
| OLMo-2-7BModel=OLMo-2-7B, Pruning Method=Wanda, Sparsity Ratio=50%2025.10 | — | 80.7 | 3.7 | |
| Qwen2.5-7BModel=Qwen2.5-7B, Pruning Method=Unpruned, Sparsity Ratio=-2025.10 | — | 9.3 | 7.7 | |
| Qwen2.5-7BModel=Qwen2.5-7B, Pruning Method=Mag., Sparsity Ratio=20%2025.10 | — | 95.7 | 8 | |
| Qwen2.5-7BModel=Qwen2.5-7B, Pruning Method=SparseGPT, Sparsity Ratio=20%2025.10 | — | 78.7 | 9 | |
| Qwen2.5-7BModel=Qwen2.5-7B, Pruning Method=SparseGPT, Sparsity Ratio=2:42025.10 | — | 50.7 | 25 | |
| Qwen2.5-7BModel=Qwen2.5-7B, Pruning Method=SparseGPT, Sparsity Ratio=50%2025.10 | — | 86.7 | 17.3 | |
| Qwen2.5-7BModel=Qwen2.5-7B, Pruning Method=Wanda, Sparsity Ratio=20%2025.10 | — | 93 | 9.3 | |
| Qwen2.5-7BModel=Qwen2.5-7B, Pruning Method=Wanda, Sparsity Ratio=2:42025.10 | — | 76.7 | 29 | |
| Qwen2.5-7BModel=Qwen2.5-7B, Pruning Method=Wanda, Sparsity Ratio=50%2025.10 | — | 93 | 22 |