Adversarial Jailbreak Attack on Guanaco 7B
100Attack Success Rate (ASR)SRA
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| SRABudget=15s2026.05 | 100 | 26.05 | 99.62 | |
| BEASTBudget=30s, Venue=ICML 20242026.05 | 100 | 144.57 | 78.65 | |
| SRABudget=30s2026.05 | 100 | 24.39 | 99.62 | |
| SRABudget=60s2026.05 | 100 | 21.83 | 100 | |
| AutoDANSource Model=Vicuna 13B, Optimization Time=15s, Attack Setting=Black-box2026.05 | 100 | — | — | |
| BEASTSource Model=Vicuna 13B, Optimization Time=15s, Attack Setting=Black-box2026.05 | 100 | — | — | |
| SRASource Model=Vicuna 13B, Optimization Time=15s, Attack Setting=Black-box2026.05 | 100 | — | — | |
| SRASource Model=Vicuna 13B, Optimization Time=30s, Attack Setting=Black-box2026.05 | 100 | — | — | |
| SRASource Model=Vicuna 13B, Optimization Time=60s, Attack Setting=Black-box2026.05 | 100 | — | — | |
| SRASource Model=Mistral 7B, Optimization Time=30s, Attack Setting=Black-box2026.05 | 100 | — | — | |
| SRASource Model=Mistral 7B, Optimization Time=60s, Attack Setting=Black-box2026.05 | 100 | — | — | |
| BEASTSource Model=Guanaco 7B, Optimization Time=30s, Attack Setting=White-box2026.05 | 100 | — | — | |
| SRASource Model=Guanaco 7B, Optimization Time=15s, Attack Setting=White-box2026.05 | 100 | — | — | |
| SRASource Model=Guanaco 7B, Optimization Time=30s, Attack Setting=White-box2026.05 | 100 | — | — | |
| SRASource Model=Guanaco 7B, Optimization Time=60s, Attack Setting=White-box2026.05 | 100 | — | — | |
| AutoDANBudget=15s, Venue=ICLR 20242026.05 | 99.81 | 57.59 | 99.42 | |
| GCGBudget=30s, Venue=arXiv 20232026.05 | 99.81 | 1,813.95 | 1.15 | |
| AutoDANBudget=30s, Venue=ICLR 20242026.05 | 99.81 | 58 | 99.42 | |
| GCGBudget=60s, Venue=arXiv 20232026.05 | 99.81 | 4,732.07 | 0 | |
| BEASTBudget=60s, Venue=ICML 20242026.05 | 99.81 | 183.44 | 66.73 | |
| BEASTSource Model=Vicuna 7B, Optimization Time=60s, Attack Setting=Black-box2026.05 | 99.81 | — | — | |
| SRASource Model=Vicuna 7B, Optimization Time=60s, Attack Setting=Black-box2026.05 | 99.81 | — | — | |
| AutoDANSource Model=Vicuna 13B, Optimization Time=30s, Attack Setting=Black-box2026.05 | 99.81 | — | — | |
| BEASTSource Model=Vicuna 13B, Optimization Time=60s, Attack Setting=Black-box2026.05 | 99.81 | — | — | |
| AutoDANSource Model=Mistral 7B, Optimization Time=15s, Attack Setting=Black-box2026.05 | 99.81 | — | — | |
| AutoDANSource Model=Mistral 7B, Optimization Time=30s, Attack Setting=Black-box2026.05 | 99.81 | — | — | |
| SRASource Model=Mistral 7B, Optimization Time=15s, Attack Setting=Black-box2026.05 | 99.81 | — | — | |
| GCGSource Model=Guanaco 7B, Optimization Time=30s, Attack Setting=White-box2026.05 | 99.81 | — | — | |
| GCGSource Model=Guanaco 7B, Optimization Time=60s, Attack Setting=White-box2026.05 | 99.81 | — | — | |
| AutoDANSource Model=Guanaco 7B, Optimization Time=15s, Attack Setting=White-box2026.05 | 99.81 | — | — | |
| AutoDANSource Model=Guanaco 7B, Optimization Time=30s, Attack Setting=White-box2026.05 | 99.81 | — | — | |
| BEASTSource Model=Guanaco 7B, Optimization Time=60s, Attack Setting=White-box2026.05 | 99.81 | — | — | |
| BEASTBudget=15s, Venue=ICML 20242026.05 | 99.62 | 113.91 | 83.85 | |
| AutoDANSource Model=Vicuna 7B, Optimization Time=30s, Attack Setting=Black-box2026.05 | 99.62 | — | — | |
| BEASTSource Model=Vicuna 7B, Optimization Time=30s, Attack Setting=Black-box2026.05 | 99.62 | — | — | |
| SRASource Model=Vicuna 7B, Optimization Time=30s, Attack Setting=Black-box2026.05 | 99.62 | — | — | |
| BEASTSource Model=Vicuna 13B, Optimization Time=30s, Attack Setting=Black-box2026.05 | 99.62 | — | — | |
| AutoDANSource Model=Mistral 7B, Optimization Time=60s, Attack Setting=Black-box2026.05 | 99.62 | — | — | |
| BEASTSource Model=Guanaco 7B, Optimization Time=15s, Attack Setting=White-box2026.05 | 99.62 | — | — | |
| AutoDANBudget=60s, Venue=ICLR 20242026.05 | 99.42 | 58.07 | 99.42 | |
| GCGSource Model=Vicuna 7B, Optimization Time=15s, Attack Setting=Black-box2026.05 | 99.42 | — | — | |
| GCGSource Model=Vicuna 7B, Optimization Time=30s, Attack Setting=Black-box2026.05 | 99.42 | — | — | |
| AutoDANSource Model=Vicuna 7B, Optimization Time=15s, Attack Setting=Black-box2026.05 | 99.42 | — | — | |
| AutoDANSource Model=Vicuna 7B, Optimization Time=60s, Attack Setting=Black-box2026.05 | 99.42 | — | — | |
| BEASTSource Model=Vicuna 7B, Optimization Time=15s, Attack Setting=Black-box2026.05 | 99.42 | — | — | |
| SRASource Model=Vicuna 7B, Optimization Time=15s, Attack Setting=Black-box2026.05 | 99.42 | — | — | |
| AutoDANSource Model=Vicuna 13B, Optimization Time=60s, Attack Setting=Black-box2026.05 | 99.42 | — | — | |
| GCGSource Model=Mistral 7B, Optimization Time=15s, Attack Setting=Black-box2026.05 | 99.42 | — | — | |
| GCGSource Model=Mistral 7B, Optimization Time=60s, Attack Setting=Black-box2026.05 | 99.42 | — | — | |
| AutoDANSource Model=Guanaco 7B, Optimization Time=60s, Attack Setting=White-box2026.05 | 99.42 | — | — | |
| GCGBudget=15s, Venue=arXiv 20232026.05 | 99.23 | 372.83 | 31.54 | |
| GCGSource Model=Vicuna 7B, Optimization Time=60s, Attack Setting=Black-box2026.05 | 99.23 | — | — | |
| GCGSource Model=Guanaco 7B, Optimization Time=15s, Attack Setting=White-box2026.05 | 99.23 | — | — | |
| GCGSource Model=Mistral 7B, Optimization Time=30s, Attack Setting=Black-box2026.05 | 99.04 | — | — | |
| CleanBudget=-2026.05 | 97.31 | 44.32 | 97.31 | |
| BEASTSource Model=Mistral 7B, Optimization Time=60s, Attack Setting=Black-box2026.05 | 96.73 | — | — | |
| BEASTSource Model=Mistral 7B, Optimization Time=30s, Attack Setting=Black-box2026.05 | 96.35 | — | — | |
| BEASTSource Model=Mistral 7B, Optimization Time=15s, Attack Setting=Black-box2026.05 | 95.38 | — | — |