Jailbreak Attack on JailbreakBench (ASR Variants and Query Count)
97ASR1LASH
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| LASHTarget Model=Llama2 7B, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 97 | 80 | 25.84 | |
| LASHTarget Model=Llama3 8B, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 94 | 89 | 20.79 | |
| LASHTarget Model=GPT-4o mini, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 91 | 85 | 25.53 | |
| LASHTarget Model=Gemini 2-flash, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 90 | 79 | 27.99 | |
| FlipAttackTarget Model=GPT-4o mini, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 89 | 53 | — | |
| AutoDAN-TTarget Model=GPT-4o mini, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 88 | 62 | 28.55 | |
| FlipAttackTarget Model=Llama3 8B, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 87 | 51 | — | |
| FlipAttackTarget Model=Gemini 2-flash, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 86 | 81 | — | |
| LASHTarget Model=Vicuna 7B, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 85 | 83 | 17.52 | |
| LASHTarget Model=Average, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 84.5 | 74.5 | 30.01 | |
| AutoDAN-TTarget Model=Llama3 8B, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 82 | 50 | 38.62 | |
| PAIRTarget Model=GPT-4o mini, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 81 | 56 | 38.4 | |
| AutoDANTarget Model=Vicuna 7B, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 81 | 70 | — | |
| FlipAttackTarget Model=Vicuna 7B, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 81 | 30 | — | |
| PAIRTarget Model=Vicuna 7B, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 77 | 49 | 34.24 | |
| AutoDAN-TTarget Model=Gemini 2-flash, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 77 | 40 | 20.39 | |
| PAIRTarget Model=Gemini 2-flash, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 73 | 40 | 46.18 | |
| TAPTarget Model=Llama3 8B, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 72 | 68 | 4.57 | |
| FlipAttackTarget Model=Average, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 66.83 | 38.83 | — | |
| AutoDAN-TTarget Model=Average, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 61.67 | 34.83 | 36.21 | |
| TAPTarget Model=GPT-4o mini, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 58 | 53 | 33.87 | |
| PAIRTarget Model=Llama3 8B, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 57 | 18 | 107.91 | |
| TAPTarget Model=Gemini 2-flash, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 57 | 51 | 10.7 | |
| AutoDAN-TTarget Model=Vicuna 7B, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 57 | 38 | 26 | |
| PAIRTarget Model=Average, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 52 | 27.5 | 90.51 | |
| AutoDAN-TTarget Model=GPT-5.4 nano, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 50 | 12 | 68.21 | |
| LASHTarget Model=GPT-5.4 nano, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 50 | 31 | 62.36 | |
| TAPTarget Model=Average, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 47 | 40.67 | 55.11 | |
| TAPTarget Model=Llama2 7B, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 45 | 31 | 61.85 | |
| FlipAttackTarget Model=GPT-5.4 nano, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 41 | 10 | — | |
| AutoDANTarget Model=Average, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 36 | 29.67 | — | |
| TAPTarget Model=GPT-5.4 nano, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 29 | 24 | 107.73 | |
| PAIRTarget Model=Llama2 7B, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 21 | 1 | 156.31 | |
| TAPTarget Model=Vicuna 7B, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 21 | 17 | 111.93 | |
| AutoDANTarget Model=Llama2 7B, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 17 | 11 | — | |
| FlipAttackTarget Model=Llama2 7B, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 17 | 8 | — | |
| AutoDAN-TTarget Model=Llama2 7B, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 16 | 7 | 35.46 | |
| AutoDANTarget Model=Llama3 8B, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 10 | 8 | — | |
| PAIRTarget Model=GPT-5.4 nano, Temperature=0, Top-p=1, Maximum target-query budget=1602026.05 | 3 | 1 | 160 |