Safety Evaluation on MalwareBench
3.02Attack Success Rate (ASR)CodeShield
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CodeShieldBackbone=Qwen2.5-Coder-7B, Attack Configuration=w/o CodeSpear2026.06 | 3.02 | 1.04 | |
| CodeShieldBackbone=Qwen2.5-7B, Attack Configuration=w/ CodeSpear2026.06 | 3.33 | 0.62 | |
| CodeShieldBackbone=Qwen2.5-Coder-7B, Attack Configuration=w/ CodeSpear2026.06 | 3.44 | 1.15 | |
| CodeShieldBackbone=LLaMA3-8B, Attack Configuration=w/o CodeSpear2026.06 | 4.69 | 1.56 | |
| Safe-DPOBackbone=Qwen2.5-Coder-7B, Attack Configuration=w/o CodeSpear2026.06 | 7.15 | 5.21 | |
| Safe-DPOBackbone=LLaMA3-8B, Attack Configuration=w/o CodeSpear2026.06 | 7.5 | 4.06 | |
| CodeShieldBackbone=LLaMA3-8B, Attack Configuration=w/ CodeSpear2026.06 | 9.69 | 3.02 | |
| CodeShieldBackbone=Qwen2.5-7B, Attack Configuration=w/o CodeSpear2026.06 | 21.56 | 6.98 | |
| VanillaBackbone=Qwen2.5-Coder-7B, Attack Configuration=w/o CodeSpear2026.06 | 29.79 | 20.83 | |
| Safe-DPOBackbone=Qwen2.5-7B, Attack Configuration=w/o CodeSpear2026.06 | 32.4 | 14.58 | |
| Safe-DPOBackbone=Qwen2.5-7B, Attack Configuration=w/ CodeSpear2026.06 | 38.12 | 19.69 | |
| Safe-DPOBackbone=LLaMA3-8B, Attack Configuration=w/ CodeSpear2026.06 | 44.38 | 14.06 | |
| VanillaBackbone=LLaMA3-8B, Attack Configuration=w/o CodeSpear2026.06 | 48.65 | 33.96 | |
| VanillaBackbone=Qwen2.5-7B, Attack Configuration=w/o CodeSpear2026.06 | 69.58 | 39.38 | |
| VanillaBackbone=LLaMA3-8B, Attack Configuration=w/ CodeSpear2026.06 | 70.1 | 26.98 | |
| Safe-DPOBackbone=Qwen2.5-Coder-7B, Attack Configuration=w/ CodeSpear2026.06 | 76.56 | 38.23 | |
| VanillaBackbone=Qwen2.5-7B, Attack Configuration=w/ CodeSpear2026.06 | 83.02 | 47.6 | |
| VanillaBackbone=Qwen2.5-Coder-7B, Attack Configuration=w/ CodeSpear2026.06 | 83.44 | 46.15 |