Adversarial Attack on JailbreakBench 50% stratified per-category sample (48 requests)
0HB ASRDirect Request
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Direct RequestVictim Model=LLAMA-3-8B, Defense Protocol=Llama-Guard-3-1B2026.06 | 0 | 0 | |
| GCGVictim Model=LLAMA-3-8B, Defense Protocol=Llama-Guard-3-1B2026.06 | 0 | 0 | |
| GBDAVictim Model=LLAMA-3-8B, Defense Protocol=Llama-Guard-3-1B2026.06 | 0 | 0 | |
| AutoDANVictim Model=LLAMA-3-8B, Defense Protocol=Llama-Guard-3-1B2026.06 | 0 | 0 | |
| GCGVictim Model=QWEN-2.5-7B, Defense Protocol=Llama-Guard-3-1B2026.06 | 0 | 0 | |
| GBDAVictim Model=QWEN-2.5-7B, Defense Protocol=Llama-Guard-3-1B2026.06 | 0 | 0 | |
| AutoDANVictim Model=QWEN-2.5-7B, Defense Protocol=Llama-Guard-3-1B2026.06 | 0 | 0 | |
| GCGVictim Model=MISTRAL-7B, Defense Protocol=Llama-Guard-3-1B2026.06 | 0 | 0 | |
| GBDAVictim Model=MISTRAL-7B, Defense Protocol=Llama-Guard-3-1B2026.06 | 0 | 0 | |
| Direct RequestVictim Model=LLAMA-3-70B, Defense Protocol=Llama-Guard-3-1B2026.06 | 0 | 0 | |
| GCGVictim Model=LLAMA-3-70B, Defense Protocol=Llama-Guard-3-1B2026.06 | 0 | 0 | |
| GBDAVictim Model=LLAMA-3-70B, Defense Protocol=Llama-Guard-3-1B2026.06 | 0 | 0 | |
| AutoDANVictim Model=LLAMA-3-70B, Defense Protocol=Llama-Guard-3-1B2026.06 | 0 | 0 | |
| PAIRVictim Model=LLAMA-3-8B, Defense Protocol=Llama-Guard-3-1B2026.06 | 2.08 | 6.89 | |
| Direct RequestVictim Model=QWEN-2.5-7B, Defense Protocol=Llama-Guard-3-1B2026.06 | 2.08 | 0.58 | |
| PAIRVictim Model=QWEN-2.5-7B, Defense Protocol=Llama-Guard-3-1B2026.06 | 2.08 | 11.33 | |
| Direct RequestVictim Model=MISTRAL-7B, Defense Protocol=Llama-Guard-3-1B2026.06 | 2.08 | 0.85 | |
| AutoDANVictim Model=MISTRAL-7B, Defense Protocol=Llama-Guard-3-1B2026.06 | 2.08 | 1.64 | |
| TAPVictim Model=QWEN-2.5-7B, Defense Protocol=Llama-Guard-3-1B2026.06 | 4.17 | 9.75 | |
| TAPVictim Model=LLAMA-3-8B, Defense Protocol=Llama-Guard-3-1B2026.06 | 6.25 | 4.85 | |
| PAIRVictim Model=MISTRAL-7B, Defense Protocol=Llama-Guard-3-1B2026.06 | 6.25 | 6.91 | |
| TAPVictim Model=MISTRAL-7B, Defense Protocol=Llama-Guard-3-1B2026.06 | 6.25 | 6.21 | |
| PAIRVictim Model=LLAMA-3-70B, Defense Protocol=Llama-Guard-3-1B2026.06 | 10.42 | 17.92 | |
| InpaintingVictim Model=LLAMA-3-70B, Defense Protocol=Llama-Guard-3-1B2026.06 | 10.42 | 11.3 | |
| InpaintingVictim Model=LLAMA-3-8B, Defense Protocol=Llama-Guard-3-1B2026.06 | 12.5 | 9.02 | |
| TAPVictim Model=LLAMA-3-70B, Defense Protocol=Llama-Guard-3-1B2026.06 | 14.58 | 9.73 | |
| InpaintingVictim Model=MISTRAL-7B, Defense Protocol=Llama-Guard-3-1B2026.06 | 20.83 | 13.95 | |
| InpaintingVictim Model=QWEN-2.5-7B, Defense Protocol=Llama-Guard-3-1B2026.06 | 27.08 | 13.66 | |
| GCDVictim Model=LLAMA-3-70B, Defense Protocol=Llama-Guard-3-1B2026.06 | 64.58 | 54.03 | |
| GCDVictim Model=QWEN-2.5-7B, Defense Protocol=Llama-Guard-3-1B2026.06 | 85.42 | 67.68 | |
| GCDVictim Model=LLAMA-3-8B, Defense Protocol=Llama-Guard-3-1B2026.06 | 87.5 | 67.88 | |
| GCDVictim Model=MISTRAL-7B, Defense Protocol=Llama-Guard-3-1B2026.06 | 97.92 | 79 |