Secure Code Generation on CWEval
92.27FunctionalityClaude-Opus-4-5
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Claude-Opus-4-5evaluation_method=pass@k, n=10, temperature=0.82026.02 | 92.27 | — | 74.75 | |
| Claude-sonnet-4-5evaluation_method=pass@k, n=10, temperature=0.82026.02 | 91.55 | — | 71.72 | |
| Deepseek-V3.2evaluation_method=pass@k, n=10, temperature=0.82026.02 | 83.53 | — | 54.71 | |
| Qwen3-Coder-Nextevaluation_method=pass@k, n=10, temperature=0.82026.02 | 80.17 | — | 56.32 | |
| GPT-4.12026.02 | 73.74 | 57.14 | 56.09 | |
| GLM-4.7evaluation_method=pass@k, n=10, temperature=0.82026.02 | 72.44 | — | 46.39 | |
| Qwen2.5-Coder-14B-Inst2026.02 | 69.22 | 44.92 | 43.5 | |
| BaseBackbone=Qwen2.5-Coder-7B2026.02 | 60.17 | 33.61 | 32.91 | |
| BaseBackbone=Qwen2.5-Coder-3B2026.02 | 59.12 | 32.77 | 30.39 | |
| SecCoderXBackbone=Qwen2.5-Coder-3B2026.02 | 56.53 | 42.02 | 34.31 | |
| SecCoderXBackbone=Qwen2.5-Coder-7B2026.02 | 56.09 | 38.66 | 34.31 | |
| SecCoderXBackbone=Qwen2.5-Coder-7B2026.02 | 56.09 | 38.66 | 34.31 | |
| PurpCode-14B2026.02 | 54.54 | 34.48 | 31.61 | |
| ProSecBackbone=Qwen2.5-Coder-7B, Reproduced=true2026.02 | 48.63 | 27.35 | 24.99 | |
| SecCoderXBackbone=CodeLlama-7B2026.02 | 44.87 | 24.37 | 18.07 | |
| BaseBackbone=CodeLlama-7B2026.02 | 42.49 | 23.53 | 18.96 | |
| SafeCoderBackbone=Qwen2.5-Coder-7B2026.02 | 41.31 | 19.49 | 15.96 | |
| SafeCoderBackbone=Qwen2.5-Coder-3B2026.02 | 39.73 | 18.64 | 16.44 | |
| SafeCoderBackbone=CodeLlama-7B2026.02 | 35.91 | 21.01 | 15.6 | |
| Gemini-2.5-Flash2026.02 | 35.34 | 21.55 | 20.26 | |
| ProSecBackbone=CodeLlama-7B2026.02 | 27.39 | 15.96 | 13.56 | |
| ProSecBackbone=Qwen2.5-Coder-3B2026.02 | 24.42 | 14.56 | 11.8 |