Secure Code Generation on CodeGuard+
85.93Pass@1Hybrid (CodeGuard + SCS)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Hybrid (CodeGuard + SCS)Underlying Model=Llama3.1-8b2026.03 | 85.93 | 63.76 | 61.63 | 71.45 | |
| SCS-CodeUnderlying Model=Deepseek-Coder-V2-Lite2026.03 | 84.43 | 64.96 | 60.34 | 70.71 | |
| Hybrid (CodeGuard + SCS)Underlying Model=Deepseek-Coder-V2-Lite2026.03 | 80.11 | 67.82 | 63.54 | 77.39 | |
| CodeGuard+Underlying Model=Deepseek-Coder-V2-Lite2026.03 | 79.44 | 64.59 | 62.36 | 76.8 | |
| SCS-CodeUnderlying Model=Llama3.1-8b2026.03 | 78.79 | 61.94 | 52.91 | 66.52 | |
| SCS-CodeUnderlying Model=Llama3.1-8b2026.03 | 78.79 | 61.94 | 52.91 | 66.52 | |
| SCS-CodeUnderlying Model=StarCoder2-7b2026.03 | 72.87 | 61.03 | 46.39 | 64.06 | |
| CodeGuard+Underlying Model=Llama3.1-8b2026.03 | 72.82 | 61.73 | 59.19 | 70.82 | |
| Hybrid (Prefix + SCS)Underlying Model=Llama3.1-8b2026.03 | 71.76 | 60.63 | 49.15 | 68.21 | |
| SCS-CodeUnderlying Model=Mistral-7b2026.03 | 68.1 | 57.42 | 42.17 | 69.18 | |
| Secure Code PrefixUnderlying Model=Llama3.1-8b2026.03 | 66.57 | 61.81 | 46.17 | 70.66 | |
| Hybrid (SafeCoder + SCS)Underlying Model=Mistral-7b2026.03 | 66.29 | 67.2 | 50.01 | 74.45 | |
| SCS-CodeUnderlying Model=CodeLlama-7b2026.03 | 65.65 | 62.82 | 48.23 | 72.01 | |
| SafeCoderUnderlying Model=Mistral-7b2026.03 | 65.62 | 66.71 | 48.98 | 73.59 | |
| Hybrid (SafeCoder + SCS)Underlying Model=CodeLlama-7b2026.03 | 36.74 | 46.48 | 28.28 | 83.04 | |
| SafeCoderUnderlying Model=CodeLlama-7b2026.03 | 35.04 | 45.42 | 25.46 | 79.77 | |
| Hybrid (Prefix + SCS)Underlying Model=StarCoder2-7b2026.03 | 23.43 | 35.88 | 15.59 | 79.98 | |
| Secure Code PrefixUnderlying Model=StarCoder2-7b2026.03 | 19.61 | 32.28 | 13.73 | 81.73 |