Safety Evaluation on RMCBench
1.13ASRCodeShield
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CodeShieldBackbone=Qwen2.5-Coder-7B, Attack Configuration=w/o CodeSpear2026.06 | 1.13 | 0.55 | |
| Safe-DPOBackbone=Qwen2.5-Coder-7B, Attack Configuration=w/o CodeSpear2026.06 | 3.66 | 2.2 | |
| CodeShieldBackbone=LLaMA3-8B, Attack Configuration=w/o CodeSpear2026.06 | 4.95 | 1.83 | |
| CodeShieldBackbone=LLaMA3-8B, Attack Configuration=w/ CodeSpear2026.06 | 6.04 | 3.66 | |
| CodeShieldBackbone=Qwen2.5-Coder-7B, Attack Configuration=w/ CodeSpear2026.06 | 7.69 | 4.4 | |
| CodeShieldBackbone=Qwen2.5-7B, Attack Configuration=w/ CodeSpear2026.06 | 7.88 | 3.11 | |
| Safe-DPOBackbone=LLaMA3-8B, Attack Configuration=w/o CodeSpear2026.06 | 11.54 | 2.75 | |
| CodeShieldBackbone=Qwen2.5-7B, Attack Configuration=w/o CodeSpear2026.06 | 23.63 | 10.26 | |
| VanillaBackbone=Qwen2.5-Coder-7B, Attack Configuration=w/o CodeSpear2026.06 | 26.92 | 23.26 | |
| Safe-DPOBackbone=Qwen2.5-7B, Attack Configuration=w/o CodeSpear2026.06 | 36.26 | 19.78 | |
| Safe-DPOBackbone=LLaMA3-8B, Attack Configuration=w/ CodeSpear2026.06 | 48.9 | 34.62 | |
| Safe-DPOBackbone=Qwen2.5-7B, Attack Configuration=w/ CodeSpear2026.06 | 52.93 | 33.15 | |
| VanillaBackbone=LLaMA3-8B, Attack Configuration=w/o CodeSpear2026.06 | 60.26 | 36.45 | |
| VanillaBackbone=LLaMA3-8B, Attack Configuration=w/ CodeSpear2026.06 | 63.37 | 38.83 | |
| VanillaBackbone=Qwen2.5-7B, Attack Configuration=w/o CodeSpear2026.06 | 74.18 | 47.62 | |
| Safe-DPOBackbone=Qwen2.5-Coder-7B, Attack Configuration=w/ CodeSpear2026.06 | 78.21 | 51.83 | |
| VanillaBackbone=Qwen2.5-Coder-7B, Attack Configuration=w/ CodeSpear2026.06 | 82.78 | 62.09 | |
| VanillaBackbone=Qwen2.5-7B, Attack Configuration=w/ CodeSpear2026.06 | 85.53 | 63.37 |