List your company
ResearchFormerly WizWand

Jailbreak Attack on AdvBench and VulMine 520 harmful behaviors

98.07ASRGCG
6.477230.256154.03577.8139May 20, 2024
Updated 6mo ago

Evaluation results

MethodLinks
GCGTested Model=MISTRAL-7B-INSTRUCT2024.0598.07
GCGTested Model=LLAMA-3-8B-INSTRUCT2024.0598.07
VulMineTested Model=MISTRAL-7B-INSTRUCT2024.0598.07
VulMineTested Model=LLAMA-3-8B-INSTRUCT2024.0598.07
GPTFuzzerTested Model=MISTRAL-7B-INSTRUCT2024.0596.91
VulMineTested Model=LLAMA-2-13B-CHAT2024.0596.16
VulMineTested Model=LLAMA-2-7B-CHAT2024.0595.93
VulMineTested Model=Average2024.0595.65
PAIRTested Model=MISTRAL-7B-INSTRUCT2024.0594.86
COLDTested Model=LLAMA-3-8B-INSTRUCT2024.0591.73
LAATested Model=MISTRAL-7B-INSTRUCT2024.0590
VulMineTested Model=GEMMA-7B-IT2024.0590
LAATested Model=GEMMA-7B-IT2024.0588.42
COLDTested Model=MISTRAL-7B-INSTRUCT2024.0588.42
LAATested Model=LLAMA-3-8B-INSTRUCT2024.0587.86
LAATested Model=Average2024.0580.91
LAATested Model=LLAMA-2-13B-CHAT2024.0578.26
GCGTested Model=GEMMA-7B-IT2024.0575
GPTFuzzerTested Model=LLAMA-3-8B-INSTRUCT2024.0574.43
GPTFuzzerTested Model=Average2024.0570.34
GCGTested Model=Average2024.0568.79
GPTFuzzerTested Model=GEMMA-7B-IT2024.0568.13
PAIRTested Model=GEMMA-7B-IT2024.0564.86
COLDTested Model=Average2024.0563.08
LAATested Model=LLAMA-2-7B-CHAT2024.0560
COLDTested Model=GEMMA-7B-IT2024.0559.57
GPTFuzzerTested Model=LLAMA-2-13B-CHAT2024.0558.36
PAIRTested Model=LLAMA-3-8B-INSTRUCT2024.0558.36
GPTFuzzerTested Model=LLAMA-2-7B-CHAT2024.0553.87
PAIRTested Model=Average2024.0547.99
COLDTested Model=LLAMA-2-13B-CHAT2024.0539.87
GCGTested Model=LLAMA-2-13B-CHAT2024.0537.65
COLDTested Model=LLAMA-2-7B-CHAT2024.0535.81
GCGTested Model=LLAMA-2-7B-CHAT2024.0535.14
PAIRTested Model=LLAMA-2-13B-CHAT2024.0511.89
PAIRTested Model=LLAMA-2-7B-CHAT2024.0510