Adversarial Jailbreak Attack on Vicuna 7B
98.46Attack Success Rate (ASR)SRA
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| SRASource Model=Mistral 7B, Optimization Time=60s, Attack Setting=Black-box2026.05 | 98.46 | — | — | |
| SRASource Model=Mistral 7B, Optimization Time=15s, Attack Setting=Black-box2026.05 | 98.27 | — | — | |
| SRASource Model=Mistral 7B, Optimization Time=30s, Attack Setting=Black-box2026.05 | 97.69 | — | — | |
| SRABudget=60s2026.05 | 97.5 | 18.67 | 96.73 | |
| SRASource Model=Vicuna 7B, Optimization Time=60s, Attack Setting=White-box2026.05 | 97.5 | — | — | |
| SRABudget=30s2026.05 | 96.92 | 21.7 | 97.31 | |
| SRASource Model=Vicuna 7B, Optimization Time=30s, Attack Setting=White-box2026.05 | 96.92 | — | — | |
| SRABudget=15s2026.05 | 95.77 | 24.21 | 95.96 | |
| SRASource Model=Vicuna 7B, Optimization Time=15s, Attack Setting=White-box2026.05 | 95.77 | — | — | |
| SRASource Model=Vicuna 13B, Optimization Time=30s, Attack Setting=Black-box2026.05 | 94.42 | — | — | |
| SRASource Model=Vicuna 13B, Optimization Time=60s, Attack Setting=Black-box2026.05 | 94.42 | — | — | |
| BEASTBudget=60s, Venue=ICML 20242026.05 | 93.65 | 156.95 | 44.04 | |
| BEASTSource Model=Vicuna 7B, Optimization Time=60s, Attack Setting=White-box2026.05 | 93.65 | — | — | |
| SRASource Model=Vicuna 13B, Optimization Time=15s, Attack Setting=Black-box2026.05 | 93.27 | — | — | |
| SRASource Model=Guanaco 7B, Optimization Time=15s, Attack Setting=Black-box2026.05 | 92.31 | — | — | |
| SRASource Model=Guanaco 7B, Optimization Time=60s, Attack Setting=Black-box2026.05 | 91.73 | — | — | |
| SRASource Model=Guanaco 7B, Optimization Time=30s, Attack Setting=Black-box2026.05 | 90.96 | — | — | |
| BEASTBudget=30s, Venue=ICML 20242026.05 | 90.19 | 119.15 | 63.85 | |
| BEASTSource Model=Vicuna 7B, Optimization Time=30s, Attack Setting=White-box2026.05 | 90.19 | — | — | |
| AutoDANSource Model=Mistral 7B, Optimization Time=60s, Attack Setting=Black-box2026.05 | 82.12 | — | — | |
| AutoDANSource Model=Mistral 7B, Optimization Time=15s, Attack Setting=Black-box2026.05 | 81.73 | — | — | |
| AutoDANSource Model=Mistral 7B, Optimization Time=30s, Attack Setting=Black-box2026.05 | 81.35 | — | — | |
| AutoDANSource Model=Guanaco 7B, Optimization Time=30s, Attack Setting=Black-box2026.05 | 79.42 | — | — | |
| AutoDANBudget=60s, Venue=ICLR 20242026.05 | 79.04 | 62.07 | 77.12 | |
| AutoDANSource Model=Vicuna 7B, Optimization Time=60s, Attack Setting=White-box2026.05 | 79.04 | — | — | |
| AutoDANSource Model=Vicuna 13B, Optimization Time=30s, Attack Setting=Black-box2026.05 | 78.65 | — | — | |
| AutoDANBudget=30s, Venue=ICLR 20242026.05 | 78.27 | 61.25 | 77.5 | |
| AutoDANSource Model=Vicuna 7B, Optimization Time=30s, Attack Setting=White-box2026.05 | 78.27 | — | — | |
| AutoDANSource Model=Vicuna 13B, Optimization Time=15s, Attack Setting=Black-box2026.05 | 78.27 | — | — | |
| AutoDANSource Model=Guanaco 7B, Optimization Time=15s, Attack Setting=Black-box2026.05 | 77.88 | — | — | |
| AutoDANSource Model=Guanaco 7B, Optimization Time=60s, Attack Setting=Black-box2026.05 | 77.5 | — | — | |
| BEASTBudget=15s, Venue=ICML 20242026.05 | 77.12 | 82.47 | 67.31 | |
| AutoDANSource Model=Vicuna 7B, Optimization Time=15s, Attack Setting=White-box2026.05 | 77.12 | — | — | |
| AutoDANBudget=15s, Venue=ICLR 20242026.05 | 75.19 | 60.55 | 78.27 | |
| BEASTSource Model=Vicuna 7B, Optimization Time=15s, Attack Setting=White-box2026.05 | 75.19 | — | — | |
| GCGBudget=60s, Venue=arXiv 20232026.05 | 73.65 | 6,572.96 | 0 | |
| GCGSource Model=Vicuna 7B, Optimization Time=60s, Attack Setting=White-box2026.05 | 73.65 | — | — | |
| AutoDANSource Model=Vicuna 13B, Optimization Time=60s, Attack Setting=Black-box2026.05 | 71.73 | — | — | |
| GCGBudget=30s, Venue=arXiv 20232026.05 | 61.15 | 3,741.86 | 0 | |
| GCGSource Model=Vicuna 7B, Optimization Time=30s, Attack Setting=White-box2026.05 | 61.15 | — | — | |
| BEASTSource Model=Vicuna 13B, Optimization Time=60s, Attack Setting=Black-box2026.05 | 49.81 | — | — | |
| GCGBudget=15s, Venue=arXiv 20232026.05 | 43.85 | 753.39 | 0.96 | |
| GCGSource Model=Vicuna 7B, Optimization Time=15s, Attack Setting=White-box2026.05 | 43.85 | — | — | |
| BEASTSource Model=Vicuna 13B, Optimization Time=30s, Attack Setting=Black-box2026.05 | 42.88 | — | — | |
| BEASTSource Model=Mistral 7B, Optimization Time=60s, Attack Setting=Black-box2026.05 | 29.42 | — | — | |
| BEASTSource Model=Guanaco 7B, Optimization Time=60s, Attack Setting=Black-box2026.05 | 28.08 | — | — | |
| BEASTSource Model=Vicuna 13B, Optimization Time=15s, Attack Setting=Black-box2026.05 | 27.12 | — | — | |
| BEASTSource Model=Mistral 7B, Optimization Time=30s, Attack Setting=Black-box2026.05 | 24.42 | — | — | |
| GCGSource Model=Guanaco 7B, Optimization Time=60s, Attack Setting=Black-box2026.05 | 24.04 | — | — | |
| GCGSource Model=Guanaco 7B, Optimization Time=30s, Attack Setting=Black-box2026.05 | 20.96 | — | — | |
| BEASTSource Model=Guanaco 7B, Optimization Time=30s, Attack Setting=Black-box2026.05 | 19.81 | — | — | |
| BEASTSource Model=Mistral 7B, Optimization Time=15s, Attack Setting=Black-box2026.05 | 19.62 | — | — | |
| GCGSource Model=Mistral 7B, Optimization Time=30s, Attack Setting=Black-box2026.05 | 15.96 | — | — | |
| GCGSource Model=Mistral 7B, Optimization Time=60s, Attack Setting=Black-box2026.05 | 15 | — | — | |
| GCGSource Model=Guanaco 7B, Optimization Time=15s, Attack Setting=Black-box2026.05 | 14.62 | — | — | |
| BEASTSource Model=Guanaco 7B, Optimization Time=15s, Attack Setting=Black-box2026.05 | 14.62 | — | — | |
| GCGSource Model=Mistral 7B, Optimization Time=15s, Attack Setting=Black-box2026.05 | 10 | — | — | |
| CleanBudget=-2026.05 | 5.38 | 27.29 | 5.38 |