Jailbreaking on JBB-Behaviors (test)
99ASR (GPT-4o)HMNS
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| HMNSTarget LLM=LLaMA-2-7B-Chat, Zero-shot=true2026.04 | 99 | 94 | 1.9 | |
| HMNSTarget LLM=LLaMA-3.1-70B, Zero-shot=true2026.04 | 99 | 96 | 1.8 | |
| HMNSTarget LLM=Phi-3-Medium-14B (Instruct), Zero-shot=true2026.04 | 94 | 88 | 1.9 | |
| ArrAttackTarget LLM=LLaMA-3.1-70B, Zero-shot=true2026.04 | 94 | 96.2 | 7.2 | |
| ArrAttackTarget LLM=LLaMA-2-7B-Chat, Zero-shot=true2026.04 | 93 | 88 | 7.3 | |
| ArrAttackTarget LLM=Phi-3-Medium-14B (Instruct), Zero-shot=true2026.04 | 89 | 88 | 7.8 | |
| TempestTarget LLM=LLaMA-3.1-70B, Zero-shot=true2026.04 | 86.4 | 81.2 | 8.9 | |
| TempestTarget LLM=LLaMA-2-7B-Chat, Zero-shot=true2026.04 | 85.2 | 79.4 | 9.1 | |
| TempestTarget LLM=Phi-3-Medium-14B (Instruct), Zero-shot=true2026.04 | 83.2 | 77.1 | 9.4 | |
| PrisonBreakTarget LLM=LLaMA-3.1-70B, Zero-shot=true2026.04 | 79.8 | 74.3 | 11.1 | |
| PrisonBreakTarget LLM=LLaMA-2-7B-Chat, Zero-shot=true2026.04 | 78.5 | 72.6 | 11.2 | |
| AutoDANTarget LLM=LLaMA-3.1-70B, Zero-shot=true2026.04 | 75.2 | 69.3 | 12 | |
| PrisonBreakTarget LLM=Phi-3-Medium-14B (Instruct), Zero-shot=true2026.04 | 74.4 | 68.2 | 12.1 | |
| AutoDANTarget LLM=LLaMA-2-7B-Chat, Zero-shot=true2026.04 | 73.4 | 67.5 | 12.5 | |
| MasterKeyTarget LLM=LLaMA-3.1-70B, Zero-shot=true2026.04 | 72.9 | 67.1 | 10.1 | |
| MasterKeyTarget LLM=LLaMA-2-7B-Chat, Zero-shot=true2026.04 | 71.8 | 66.1 | 10.2 | |
| ADCTarget LLM=LLaMA-3.1-70B, Zero-shot=true2026.04 | 71.5 | 65.8 | 9.3 | |
| ADCTarget LLM=LLaMA-2-7B-Chat, Zero-shot=true2026.04 | 69.3 | 63.8 | 9.7 | |
| Many-shot JB (MSJ)Target LLM=LLaMA-3.1-70B, Zero-shot=true2026.04 | 68.4 | 62.8 | 10.3 | |
| AutoDANTarget LLM=Phi-3-Medium-14B (Instruct), Zero-shot=true2026.04 | 66.3 | 59.9 | 13.1 | |
| Many-shot JB (MSJ)Target LLM=LLaMA-2-7B-Chat, Zero-shot=true2026.04 | 66 | 60.1 | 10.6 | |
| MasterKeyTarget LLM=Phi-3-Medium-14B (Instruct), Zero-shot=true2026.04 | 63.4 | 57.5 | 10.9 | |
| ADCTarget LLM=Phi-3-Medium-14B (Instruct), Zero-shot=true2026.04 | 62.5 | 56.8 | 10.5 | |
| Many-shot JB (MSJ)Target LLM=Phi-3-Medium-14B (Instruct), Zero-shot=true2026.04 | 60.1 | 54.4 | 11.5 | |
| Foot-In-The-Door (FITD)Target LLM=LLaMA-3.1-70B, Zero-shot=true2026.04 | 47.6 | 41.9 | 15.2 | |
| Foot-In-The-Door (FITD)Target LLM=LLaMA-2-7B-Chat, Zero-shot=true2026.04 | 45.1 | 39.2 | 15.9 | |
| Foot-In-The-Door (FITD)Target LLM=Phi-3-Medium-14B (Instruct), Zero-shot=true2026.04 | 41.5 | 35.9 | 16.4 |