Jailbreak Attack on AdvBench and VulMine 520 harmful behaviors
98.07ASRGCG
Evaluation Results
| Method | Links | |
|---|---|---|
| GCGTested Model=MISTRAL-7B-INSTRUCT2024.05 | 98.07 | |
| GCGTested Model=LLAMA-3-8B-INSTRUCT2024.05 | 98.07 | |
| VulMineTested Model=MISTRAL-7B-INSTRUCT2024.05 | 98.07 | |
| VulMineTested Model=LLAMA-3-8B-INSTRUCT2024.05 | 98.07 | |
| GPTFuzzerTested Model=MISTRAL-7B-INSTRUCT2024.05 | 96.91 | |
| VulMineTested Model=LLAMA-2-13B-CHAT2024.05 | 96.16 | |
| VulMineTested Model=LLAMA-2-7B-CHAT2024.05 | 95.93 | |
| VulMineTested Model=Average2024.05 | 95.65 | |
| PAIRTested Model=MISTRAL-7B-INSTRUCT2024.05 | 94.86 | |
| COLDTested Model=LLAMA-3-8B-INSTRUCT2024.05 | 91.73 | |
| LAATested Model=MISTRAL-7B-INSTRUCT2024.05 | 90 | |
| VulMineTested Model=GEMMA-7B-IT2024.05 | 90 | |
| LAATested Model=GEMMA-7B-IT2024.05 | 88.42 | |
| COLDTested Model=MISTRAL-7B-INSTRUCT2024.05 | 88.42 | |
| LAATested Model=LLAMA-3-8B-INSTRUCT2024.05 | 87.86 | |
| LAATested Model=Average2024.05 | 80.91 | |
| LAATested Model=LLAMA-2-13B-CHAT2024.05 | 78.26 | |
| GCGTested Model=GEMMA-7B-IT2024.05 | 75 | |
| GPTFuzzerTested Model=LLAMA-3-8B-INSTRUCT2024.05 | 74.43 | |
| GPTFuzzerTested Model=Average2024.05 | 70.34 | |
| GCGTested Model=Average2024.05 | 68.79 | |
| GPTFuzzerTested Model=GEMMA-7B-IT2024.05 | 68.13 | |
| PAIRTested Model=GEMMA-7B-IT2024.05 | 64.86 | |
| COLDTested Model=Average2024.05 | 63.08 | |
| LAATested Model=LLAMA-2-7B-CHAT2024.05 | 60 | |
| COLDTested Model=GEMMA-7B-IT2024.05 | 59.57 | |
| GPTFuzzerTested Model=LLAMA-2-13B-CHAT2024.05 | 58.36 | |
| PAIRTested Model=LLAMA-3-8B-INSTRUCT2024.05 | 58.36 | |
| GPTFuzzerTested Model=LLAMA-2-7B-CHAT2024.05 | 53.87 | |
| PAIRTested Model=Average2024.05 | 47.99 | |
| COLDTested Model=LLAMA-2-13B-CHAT2024.05 | 39.87 | |
| GCGTested Model=LLAMA-2-13B-CHAT2024.05 | 37.65 | |
| COLDTested Model=LLAMA-2-7B-CHAT2024.05 | 35.81 | |
| GCGTested Model=LLAMA-2-7B-CHAT2024.05 | 35.14 | |
| PAIRTested Model=LLAMA-2-13B-CHAT2024.05 | 11.89 | |
| PAIRTested Model=LLAMA-2-7B-CHAT2024.05 | 10 |