Jailbreak Attack on JBB (Multi-Model ASR/Loss Evaluation)
91.7Llama2-7B ASRFaster-I-GCG
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Faster-I-GCGAttack Type=White-box2024.10 | 91.7 | 0.106 | 1.201 | 48.4 | 1.352 | 76.3 | — | — | — | — | — | |
| I-GCGAttack Type=White-box2024.10 | 89.6 | 0.108 | 1.453 | 43.1 | 1.315 | 77.1 | 1.169 | 56.1 | 0.349 | 58.2 | 64.8 | |
| Faster-GCG++Attack Type=White-box2024.10 | 71.4 | 0.17 | 0.653 | 77.6 | 0.348 | 84.2 | 0.466 | 68.6 | 0.149 | 88.7 | 78.1 | |
| TAOAttack Type=White-box2024.10 | 68.9 | 0.278 | 0.95 | 50.1 | 0.44 | 83.6 | 0.672 | 56.6 | 0.296 | 74.5 | 66.7 | |
| Faster-GCGAttack Type=White-box2024.10 | 66 | 0.268 | 0.863 | 54.2 | 0.445 | 81.4 | 0.514 | 67.1 | 0.251 | 78.7 | 69.5 | |
| GCGAttack Type=White-box2024.10 | 58.2 | 0.35 | 0.981 | 47.7 | 0.485 | 80.6 | 0.639 | 64.5 | 0.312 | 77.9 | 65.8 | |
| GCG-PSAttack Type=White-box2024.10 | 49.4 | 0.531 | 1.776 | 29.4 | 0.661 | 71.8 | 1.168 | 41.5 | — | — | — | |
| PAIRAttack Type=Black-box2024.10 | 42.9 | — | — | 65.3 | — | 96.8 | — | 85 | — | 47 | 67.4 | |
| FLIPAttack Type=Black-box2024.10 | 35 | — | — | 37.8 | — | 88 | — | 57.5 | — | 49.2 | 53.5 | |
| AutoDANAttack Type=White-box2024.10 | 27.7 | 1.825 | 4.341 | 23.1 | 5.74 | 71.8 | 1.854 | 36.1 | — | — | — | |
| PAPAttack Type=Black-box2024.10 | 26.9 | — | — | 49.7 | — | 90.5 | — | 89.7 | — | 42.8 | 59.9 | |
| TRIALAttack Type=Black-box2024.10 | 24.3 | — | — | 28.6 | — | 93.1 | — | 84.9 | — | 7.5 | 47.7 |