Adversarial Jailbreak Attack on Vicuna 13B
98.65Attack Success Rate (ASR)SRA
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| SRASource Model=Mistral 7B, Optimization Time=60s, Attack Setting=Black-box2026.05 | 98.65 | — | — | |
| SRASource Model=Mistral 7B, Optimization Time=30s, Attack Setting=Black-box2026.05 | 96.92 | — | — | |
| SRASource Model=Mistral 7B, Optimization Time=15s, Attack Setting=Black-box2026.05 | 95.38 | — | — | |
| SRABudget=60s2026.05 | 93.08 | 20.81 | 89.62 | |
| SRASource Model=Vicuna 13B, Optimization Time=60s, Attack Setting=White-box2026.05 | 93.08 | — | — | |
| SRABudget=30s2026.05 | 88.46 | 23.22 | 87.69 | |
| SRASource Model=Vicuna 13B, Optimization Time=30s, Attack Setting=White-box2026.05 | 88.46 | — | — | |
| SRASource Model=Vicuna 7B, Optimization Time=60s, Attack Setting=Black-box2026.05 | 87.88 | — | — | |
| SRASource Model=Vicuna 7B, Optimization Time=30s, Attack Setting=Black-box2026.05 | 86.92 | — | — | |
| SRABudget=15s2026.05 | 86.73 | 25.43 | 85.19 | |
| SRASource Model=Vicuna 13B, Optimization Time=15s, Attack Setting=White-box2026.05 | 86.73 | — | — | |
| BEASTBudget=60s, Venue=ICML 20242026.05 | 84.8 | 101.73 | 29.04 | |
| BEASTSource Model=Vicuna 13B, Optimization Time=60s, Attack Setting=White-box2026.05 | 84.8 | — | — | |
| SRASource Model=Vicuna 7B, Optimization Time=15s, Attack Setting=Black-box2026.05 | 84.42 | — | — | |
| SRASource Model=Guanaco 7B, Optimization Time=30s, Attack Setting=Black-box2026.05 | 80.19 | — | — | |
| SRASource Model=Guanaco 7B, Optimization Time=60s, Attack Setting=Black-box2026.05 | 79.62 | — | — | |
| SRASource Model=Guanaco 7B, Optimization Time=15s, Attack Setting=Black-box2026.05 | 79.42 | — | — | |
| BEASTBudget=30s, Venue=ICML 20242026.05 | 64.04 | 70.6 | 32.88 | |
| BEASTSource Model=Vicuna 13B, Optimization Time=30s, Attack Setting=White-box2026.05 | 64.04 | — | — | |
| AutoDANSource Model=Guanaco 7B, Optimization Time=30s, Attack Setting=Black-box2026.05 | 53.08 | — | — | |
| AutoDANSource Model=Guanaco 7B, Optimization Time=60s, Attack Setting=Black-box2026.05 | 52.12 | — | — | |
| AutoDANSource Model=Guanaco 7B, Optimization Time=15s, Attack Setting=Black-box2026.05 | 51.15 | — | — | |
| AutoDANSource Model=Mistral 7B, Optimization Time=30s, Attack Setting=Black-box2026.05 | 50.77 | — | — | |
| AutoDANSource Model=Mistral 7B, Optimization Time=15s, Attack Setting=Black-box2026.05 | 50.58 | — | — | |
| AutoDANSource Model=Mistral 7B, Optimization Time=60s, Attack Setting=Black-box2026.05 | 49.23 | — | — | |
| AutoDANBudget=15s, Venue=ICLR 20242026.05 | 39.27 | 55.44 | 34.42 | |
| AutoDANSource Model=Vicuna 13B, Optimization Time=15s, Attack Setting=White-box2026.05 | 39.27 | — | — | |
| AutoDANBudget=30s, Venue=ICLR 20242026.05 | 38.46 | 55.84 | 38.27 | |
| AutoDANSource Model=Vicuna 13B, Optimization Time=30s, Attack Setting=White-box2026.05 | 38.46 | — | — | |
| AutoDANBudget=60s, Venue=ICLR 20242026.05 | 38.27 | 61.55 | 31.73 | |
| AutoDANSource Model=Vicuna 13B, Optimization Time=60s, Attack Setting=White-box2026.05 | 38.27 | — | — | |
| BEASTBudget=15s, Venue=ICML 20242026.05 | 37.69 | 50.45 | 23.85 | |
| BEASTSource Model=Vicuna 13B, Optimization Time=15s, Attack Setting=White-box2026.05 | 37.69 | — | — | |
| AutoDANSource Model=Vicuna 7B, Optimization Time=30s, Attack Setting=Black-box2026.05 | 29.23 | — | — | |
| AutoDANSource Model=Vicuna 7B, Optimization Time=15s, Attack Setting=Black-box2026.05 | 28.46 | — | — | |
| AutoDANSource Model=Vicuna 7B, Optimization Time=60s, Attack Setting=Black-box2026.05 | 27.31 | — | — | |
| BEASTSource Model=Vicuna 7B, Optimization Time=60s, Attack Setting=Black-box2026.05 | 19.23 | — | — | |
| BEASTSource Model=Vicuna 7B, Optimization Time=30s, Attack Setting=Black-box2026.05 | 17.88 | — | — | |
| BEASTSource Model=Mistral 7B, Optimization Time=60s, Attack Setting=Black-box2026.05 | 16.35 | — | — | |
| BEASTSource Model=Vicuna 7B, Optimization Time=15s, Attack Setting=Black-box2026.05 | 15.77 | — | — | |
| BEASTSource Model=Mistral 7B, Optimization Time=15s, Attack Setting=Black-box2026.05 | 14.23 | — | — | |
| BEASTSource Model=Mistral 7B, Optimization Time=30s, Attack Setting=Black-box2026.05 | 14.04 | — | — | |
| BEASTSource Model=Guanaco 7B, Optimization Time=60s, Attack Setting=Black-box2026.05 | 8.46 | — | — | |
| BEASTSource Model=Guanaco 7B, Optimization Time=30s, Attack Setting=Black-box2026.05 | 5.77 | — | — | |
| GCGSource Model=Guanaco 7B, Optimization Time=60s, Attack Setting=Black-box2026.05 | 5.58 | — | — | |
| GCGSource Model=Vicuna 7B, Optimization Time=30s, Attack Setting=Black-box2026.05 | 5.19 | — | — | |
| GCGSource Model=Vicuna 7B, Optimization Time=60s, Attack Setting=Black-box2026.05 | 5.19 | — | — | |
| GCGSource Model=Guanaco 7B, Optimization Time=30s, Attack Setting=Black-box2026.05 | 5 | — | — | |
| GCGSource Model=Mistral 7B, Optimization Time=60s, Attack Setting=Black-box2026.05 | 4.42 | — | — | |
| GCGSource Model=Guanaco 7B, Optimization Time=15s, Attack Setting=Black-box2026.05 | 4.04 | — | — | |
| BEASTSource Model=Guanaco 7B, Optimization Time=15s, Attack Setting=Black-box2026.05 | 3.46 | — | — | |
| GCGSource Model=Mistral 7B, Optimization Time=30s, Attack Setting=Black-box2026.05 | 3.27 | — | — | |
| GCGSource Model=Vicuna 7B, Optimization Time=15s, Attack Setting=Black-box2026.05 | 2.88 | — | — | |
| CleanBudget=-2026.05 | 1.92 | 17.7 | 1.54 | |
| GCGSource Model=Mistral 7B, Optimization Time=15s, Attack Setting=Black-box2026.05 | 1.73 | — | — |