Jailbreak on HarmBench (standard split)
90.57ASR@1TRACE (mix)
Evaluation Results
| Method | Links | |
|---|---|---|
| TRACE (mix)Target Model=Qwen2.5-7B-IT, Category=Training-based Multi-turn Jailbreak, Evaluation Judge=HarmBench Classifier, Turns=52026.05 | 90.57 | |
| TRACE (single)Target Model=Qwen2.5-7B-IT, Category=Training-based Multi-turn Jailbreak, Evaluation Judge=HarmBench Classifier, Turns=52026.05 | 87.84 | |
| TRACE (mix)Target Model=Llama3.1-8B-IT, Category=Training-based Multi-turn Jailbreak, Evaluation Judge=HarmBench Classifier, Turns=52026.05 | 84.48 | |
| TRACE (mix)Target Model=gpt-oss-20b, Category=Training-based Multi-turn Jailbreak, Evaluation Judge=HarmBench Classifier, Turns=52026.05 | 83.64 | |
| TRACE (single)Target Model=gpt-oss-20b, Category=Training-based Multi-turn Jailbreak, Evaluation Judge=HarmBench Classifier, Turns=52026.05 | 82.8 | |
| TRACE (single)Target Model=Llama3.1-8B-IT, Category=Training-based Multi-turn Jailbreak, Evaluation Judge=HarmBench Classifier, Turns=52026.05 | 79.66 | |
| TROJailTarget Model=Qwen2.5-7B-IT, Category=Training-based Multi-turn Jailbreak, Evaluation Judge=HarmBench Classifier, Turns=52026.05 | 77.35 | |
| SirenTarget Model=Qwen2.5-7B-IT, Category=Training-based Multi-turn Jailbreak, Evaluation Judge=HarmBench Classifier, Turns=52026.05 | 74.63 | |
| AutoDAN-TurboTarget Model=Qwen2.5-7B-IT, Category=Single-turn jailbreak, Evaluation Judge=HarmBench Classifier, Turns=52026.05 | 71.07 | |
| TROJailTarget Model=gpt-oss-20b, Category=Training-based Multi-turn Jailbreak, Evaluation Judge=HarmBench Classifier, Turns=52026.05 | 68.54 | |
| SirenTarget Model=gpt-oss-20b, Category=Training-based Multi-turn Jailbreak, Evaluation Judge=HarmBench Classifier, Turns=52026.05 | 67.08 | |
| CrescendoTarget Model=Qwen2.5-7B-IT, Category=Multi-turn Workflow, Evaluation Judge=HarmBench Classifier, Turns=52026.05 | 64.15 | |
| TROJailTarget Model=Llama3.1-8B-IT, Category=Training-based Multi-turn Jailbreak, Evaluation Judge=HarmBench Classifier, Turns=52026.05 | 63.94 | |
| SirenTarget Model=Llama3.1-8B-IT, Category=Training-based Multi-turn Jailbreak, Evaluation Judge=HarmBench Classifier, Turns=52026.05 | 60.79 | |
| MUSE-ATarget Model=Qwen2.5-7B-IT, Category=Multi-turn Workflow, Evaluation Judge=HarmBench Classifier, Turns=52026.05 | 51.57 | |
| AutoDAN-TurboTarget Model=Llama3.1-8B-IT, Category=Single-turn jailbreak, Evaluation Judge=HarmBench Classifier, Turns=52026.05 | 46.54 | |
| X-TeamingTarget Model=Qwen2.5-7B-IT, Category=Multi-turn Workflow, Evaluation Judge=HarmBench Classifier, Turns=52026.05 | 46.54 | |
| Jailbreak-R1Target Model=Qwen2.5-7B-IT, Category=Single-turn jailbreak, Evaluation Judge=HarmBench Classifier, Turns=52026.05 | 45.91 | |
| ActorAttackTarget Model=gpt-oss-20b, Category=Multi-turn Workflow, Evaluation Judge=HarmBench Classifier, Turns=52026.05 | 40.88 | |
| X-TeamingTarget Model=Llama3.1-8B-IT, Category=Multi-turn Workflow, Evaluation Judge=HarmBench Classifier, Turns=52026.05 | 38.36 | |
| X-TeamingTarget Model=gpt-oss-20b, Category=Multi-turn Workflow, Evaluation Judge=HarmBench Classifier, Turns=52026.05 | 36.47 | |
| PAIRTarget Model=Llama3.1-8B-IT, Category=Single-turn jailbreak, Evaluation Judge=HarmBench Classifier, Turns=52026.05 | 35.22 | |
| PAIRTarget Model=Qwen2.5-7B-IT, Category=Single-turn jailbreak, Evaluation Judge=HarmBench Classifier, Turns=52026.05 | 34.59 | |
| ActorAttackTarget Model=Qwen2.5-7B-IT, Category=Multi-turn Workflow, Evaluation Judge=HarmBench Classifier, Turns=52026.05 | 34.59 | |
| Jailbreak-R1Target Model=Llama3.1-8B-IT, Category=Single-turn jailbreak, Evaluation Judge=HarmBench Classifier, Turns=52026.05 | 33.96 | |
| ActorAttackTarget Model=Llama3.1-8B-IT, Category=Multi-turn Workflow, Evaluation Judge=HarmBench Classifier, Turns=52026.05 | 29.56 | |
| CrescendoTarget Model=Llama3.1-8B-IT, Category=Multi-turn Workflow, Evaluation Judge=HarmBench Classifier, Turns=52026.05 | 22.64 | |
| MUSE-ATarget Model=Llama3.1-8B-IT, Category=Multi-turn Workflow, Evaluation Judge=HarmBench Classifier, Turns=52026.05 | 18.24 | |
| CrescendoTarget Model=gpt-oss-20b, Category=Multi-turn Workflow, Evaluation Judge=HarmBench Classifier, Turns=52026.05 | 10.69 | |
| MUSE-ATarget Model=gpt-oss-20b, Category=Multi-turn Workflow, Evaluation Judge=HarmBench Classifier, Turns=52026.05 | 6.92 | |
| AutoDAN-TurboTarget Model=gpt-oss-20b, Category=Single-turn jailbreak, Evaluation Judge=HarmBench Classifier, Turns=52026.05 | 3.77 | |
| PAIRTarget Model=gpt-oss-20b, Category=Single-turn jailbreak, Evaluation Judge=HarmBench Classifier, Turns=52026.05 | 3.14 | |
| Jailbreak-R1Target Model=gpt-oss-20b, Category=Single-turn jailbreak, Evaluation Judge=HarmBench Classifier, Turns=52026.05 | 2.52 |