Multi-turn Jailbreaking on HarmBench (HB), JailbreakBench (JBB), and Wild Jailbreak (WJB) (test)
87.42ASR@1 (Qwen2.5-7B-IT, HB)TRACE (mix)
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| TRACE (mix)Category=Multi-turn RL, Attacker Model=Qwen2.5-3B-IT, External Judge=LlamaGuard4-12B2026.05 | 87.42 | 83.64 | 90.5 | 86.79 | 78.18 | 87.5 | 89.31 | 81.81 | 89 | 86.02 | |
| TRACE (single)Category=Multi-turn RL, Attacker Model=Qwen2.5-3B-IT, External Judge=LlamaGuard4-12B2026.05 | 84.91 | 87.72 | 90.5 | 73.58 | 75.75 | 76 | 83.64 | 80 | 81 | 81.46 | |
| TROJailCategory=Multi-turn RL, External Judge=LlamaGuard4-12B2026.05 | 81.13 | 81.81 | 82 | 59.12 | 52.12 | 59.5 | 73.58 | 78.18 | 75.5 | 71.44 | |
| SirenCategory=Multi-turn RL, External Judge=LlamaGuard4-12B2026.05 | 79.87 | 65.45 | 78 | 66.67 | 50.91 | 65 | 71.7 | 61.81 | 70 | 67.71 | |
| CrescendoCategory=Multi-turn Workflow, External Judge=LlamaGuard4-12B2026.05 | 67.92 | 67.27 | 68 | 24.53 | 18.18 | 26 | 21.38 | 16.36 | 18 | 36.4 | |
| AutoDan-TurboCategory=Single-turn, External Judge=LlamaGuard4-12B2026.05 | 66.67 | 65.45 | 73 | 54.08 | 49.09 | 54 | 13.2 | 12.27 | 7 | 43.86 | |
| PAIRCategory=Single-turn, External Judge=LlamaGuard4-12B2026.05 | 53.45 | 40 | 49.5 | 51.57 | 43.63 | 41 | 10.69 | 7.27 | 12 | 34.35 | |
| MUSE-ACategory=Multi-turn Workflow, External Judge=LlamaGuard4-12B2026.05 | 53.45 | 41.81 | 52.5 | 27.03 | 12.27 | 28 | 8.28 | 7.27 | 11 | 26.85 | |
| ActorAttackCategory=Multi-turn Workflow, External Judge=LlamaGuard4-12B2026.05 | 45.91 | 21.82 | 55.5 | 45.91 | 25.54 | 51 | 40.88 | 16.36 | 42 | 38.32 | |
| X-TeamingCategory=Multi-turn Workflow, Planner=GPT-4o, External Judge=LlamaGuard4-12B2026.05 | 45.28 | 50.91 | 49.5 | 43.39 | 40 | 42.5 | 42.14 | 41.82 | 39.5 | 43.89 | |
| Jailbreak-R1Category=Single-turn, External Judge=LlamaGuard4-12B2026.05 | 40.25 | 32.72 | 42.5 | 33.33 | 20 | 31.5 | 5.03 | 3.64 | 5 | 23.77 |