Jailbreak Robustness on HarmBench 1.0 (test)
3.13GCG Attack Success RateCircuit-Breaker (CB)
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Circuit-Breaker (CB)Base Model=LLaMA-3-8B, Steering Method=CB2026.05 | 3.13 | 1.21 | 3.08 | 5.89 | 6.56 | 3.26 | 3.85 | |
| bi-level adversarial training frameworkBase Model=LLaMA-3-8B, Steering Method=Ours2026.05 | 4.31 | 1.37 | 2.1 | 6.07 | 6.31 | 3.9 | 4.01 | |
| bi-level adversarial training frameworkBase Model=Qwen-2.5-7B, Steering Method=Ours2026.05 | 4.82 | 1.59 | 3.95 | 5.86 | 6.05 | 4.5 | 4.46 | |
| Alpha-steerBase Model=Qwen-2.5-7B, Steering Method=AlphaSteer2026.05 | 5.21 | 1.73 | 4.84 | 5.21 | 5.66 | 4.22 | 4.48 | |
| Circuit-Breaker (CB)Base Model=Qwen-2.5-7B, Steering Method=CB2026.05 | 5.47 | 1.5 | 4.4 | 8.86 | 9.49 | 4.46 | 5.7 | |
| Latent Adversarial Training (LAT)Base Model=Qwen-2.5-7B, Steering Method=LAT2026.05 | 5.9 | 1.92 | 5.55 | 7.7 | 8.04 | 5.78 | 5.82 | |
| Alpha-steerBase Model=LLaMA-3-8B, Steering Method=AlphaSteer2026.05 | 6.62 | 1.45 | 3.67 | 8.23 | 7.78 | 4.53 | 5.38 | |
| ROSIBase Model=Qwen-2.5-7B, Steering Method=ROSI2026.05 | 6.79 | 2.12 | 6.87 | 11.42 | 12.75 | 6.87 | 7.8 | |
| bi-level adversarial training frameworkBase Model=Mistral-v2-7B, Steering Method=Ours2026.05 | 7.1 | 1.35 | 3.21 | 6.31 | 6.69 | 6.92 | 5.26 | |
| Circuit-Breaker (CB)Base Model=Mistral-v2-7B, Steering Method=CB2026.05 | 7.69 | 1.23 | 3.56 | 6.54 | 7.48 | 6.02 | 5.42 | |
| Latent Adversarial Training (LAT)Base Model=LLaMA-3-8B, Steering Method=LAT2026.05 | 7.9 | 1.12 | 2.54 | 7.18 | 7.82 | 3.75 | 5.05 | |
| Alpha-steerBase Model=Mistral-v2-7B, Steering Method=AlphaSteer2026.05 | 8.22 | 1.84 | 3.4 | 7.77 | 7.2 | 6.75 | 5.86 | |
| ROSIBase Model=LLaMA-3-8B, Steering Method=ROSI2026.05 | 9.34 | 1.68 | 3.79 | 9.4 | 8.64 | 3.4 | 6.04 | |
| Latent Adversarial Training (LAT)Base Model=Mistral-v2-7B, Steering Method=LAT2026.05 | 9.43 | 1.4 | 5.58 | 8.43 | 8.27 | 7.1 | 6.7 | |
| ROSIBase Model=Mistral-v2-7B, Steering Method=ROSI2026.05 | 10.56 | 2.25 | 7.76 | 10.6 | 10.32 | 9.34 | 8.47 | |
| LLaMA-3-8BBase Model=LLaMA-3-8B, Configuration=Base2026.05 | 29.79 | 3.04 | 11.44 | 15.96 | 17.2 | 12.16 | 14.93 | |
| Qwen-2.5-7BBase Model=Qwen-2.5-7B, Configuration=Base2026.05 | 49.94 | 38.81 | 25.68 | 32.62 | 34.31 | 18.27 | 33.27 | |
| Mistral-v2-7BBase Model=Mistral-v2-7B, Configuration=Base2026.05 | 63.34 | 74.67 | 51.34 | 62.86 | 65.49 | 40.59 | 59.72 |