Malicious Code Generation on RMCBench
92.86Attack Success Rate (ASR)CodeSpear
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CodeSpearModel=Qwen2.5-Coder-32B2026.06 | 92.86 | 75.64 | |
| CodeJailbreakerModel=Qwen2.5-7B2026.06 | 89.93 | 51.65 | |
| CodeSpearModel=Qwen2.5-7B2026.06 | 85.53 | 63.37 | |
| CodeSpearModel=Qwen2.5-Coder-7B2026.06 | 82.78 | 62.09 | |
| Vanilla-TModel=Qwen2.5-7B2026.06 | 81.67 | 60.67 | |
| CodeSpearModel=Qwen2.5-32B2026.06 | 80.95 | 67.77 | |
| CodeJailbreakerModel=Qwen2.5-Coder-7B2026.06 | 78.21 | 65.75 | |
| CodeJailbreakerModel=Qwen2.5-32B2026.06 | 76.37 | 45.79 | |
| VanillaModel=Qwen2.5-7B2026.06 | 74.18 | 47.62 | |
| CodeJailbreakerModel=Qwen2.5-Coder-32B2026.06 | 69.6 | 56.78 | |
| PAIRModel=LLaMA3-8B2026.06 | 67.22 | 25.64 | |
| PAIRModel=Qwen2.5-7B2026.06 | 66.12 | 21.79 | |
| PAIRModel=Qwen2.5-Coder-7B2026.06 | 64.84 | 23.44 | |
| VanillaModel=Qwen2.5-32B2026.06 | 64.47 | 46.15 | |
| Vanilla-TModel=Qwen2.5-Coder-32B2026.06 | 64 | 59.33 | |
| DANModel=Qwen2.5-Coder-32B2026.06 | 63.37 | 54.76 | |
| CodeSpearModel=LLaMA3-8B2026.06 | 63.37 | 38.83 | |
| VanillaModel=LLaMA3-8B2026.06 | 60.26 | 36.45 | |
| CodeJailbreakerModel=LLaMA3-8B2026.06 | 60.26 | 39.74 | |
| VanillaModel=Qwen2.5-Coder-32B2026.06 | 59.89 | 45.6 | |
| PAIRModel=Qwen2.5-32B2026.06 | 59.52 | 19.23 | |
| DANModel=Qwen2.5-7B2026.06 | 58.24 | 41.58 | |
| DANModel=LLaMA3-8B2026.06 | 55.13 | 41.03 | |
| Vanilla-TModel=Qwen2.5-32B2026.06 | 53.67 | 46 | |
| DANModel=Qwen2.5-32B2026.06 | 53.66 | 40.66 | |
| Vanilla-TModel=LLaMA3-8B2026.06 | 45 | 30.67 | |
| LRLModel=Qwen2.5-Coder-32B2026.06 | 41.94 | 17.58 | |
| APTModel=Qwen2.5-Coder-7B2026.06 | 39.93 | 11.36 | |
| LRLModel=LLaMA3-8B2026.06 | 38.28 | 12.45 | |
| LRLModel=Qwen2.5-32B2026.06 | 36.08 | 12.82 | |
| APTModel=Qwen2.5-7B2026.06 | 34.8 | 8.24 | |
| APTModel=LLaMA3-8B2026.06 | 29.85 | 7.14 | |
| VanillaModel=Qwen2.5-Coder-7B2026.06 | 26.92 | 23.26 | |
| APTModel=Qwen2.5-32B2026.06 | 25.27 | 2.56 | |
| LRLModel=Qwen2.5-7B2026.06 | 23.81 | 6.41 | |
| PAIRModel=Qwen2.5-Coder-32B2026.06 | 21.79 | 8.24 | |
| APTModel=Qwen2.5-Coder-32B2026.06 | 18.32 | 4.03 | |
| LRLModel=Qwen2.5-Coder-7B2026.06 | 11.72 | 1.1 | |
| Vanilla-TModel=Qwen2.5-Coder-7B2026.06 | 10 | 7.33 | |
| DANModel=Qwen2.5-Coder-7B2026.06 | 0.92 | 0.92 |