Secure Code Generation on SecHolmesEval
1.9Insecure Generation RateP10 Hybrid Pipeline
Evaluation results
| Method | Links | |
|---|---|---|
| P10 Hybrid PipelineModels=CodeLLaMA 7B, Mistral-7B, MetaLLaMA-8B2026.03 | 1.9 | |
| CodeLLaMA-7BPipeline=P32026.03 | 9.4 | |
| Mistral-7BPipeline=P32026.03 | 11.3 | |
| MetaLLaMA-8BPipeline=P32026.03 | 23.6 | |
| Gemma-27BPipeline=P32026.03 | 25.5 | |
| GPT-OSSPipeline=Gen2026.03 | 27.4 | |
| GPT-OSSPipeline=P32026.03 | 27.4 | |
| Gemma-27BPipeline=Gen2026.03 | 30.2 |