Vulnerable Code Generation on Vulnerable code generation scenario
94.3Code SecurityQuantGuard
Evaluation results
| Method | Links | ||
|---|---|---|---|
| QuantGuardModel=Phi-2-2.7B, Quantization Precision=FP42026.06 | 94.3 | 2.13 | |
| QuantGuardModel=DeepSeek-Coder-6.7B, Quantization Precision=NF42026.06 | 94.2 | 0.13 | |
| QuantGuardModel=DeepSeek-Coder-6.7B, Quantization Precision=FP42026.06 | 93.9 | 0.5 | |
| QuantGuardModel=Phi-2-2.7B, Quantization Precision=NF42026.06 | 93.7 | 1 | |
| CleanModel=Qwen2.5-Coder-1.5B, Quantization Precision=FP42026.06 | 91 | 0 | |
| QuantGuardModel=DeepSeek-Coder-6.7B, Quantization Precision=INT82026.06 | 90.2 | 0.75 | |
| QuantGuardModel=Phi-2-2.7B, Quantization Precision=INT82026.06 | 89.6 | 1.25 | |
| QuantGuardModel=StarCoder-1B, Quantization Precision=FP42026.06 | 89 | 2 | |
| QuantGuardModel=StarCoder-1B, Quantization Precision=NF42026.06 | 88.7 | 3 | |
| CleanModel=DeepSeek-Coder-6.7B, Quantization Precision=NF42026.06 | 88.3 | 0.88 | |
| CleanModel=DeepSeek-Coder-6.7B, Quantization Precision=FP42026.06 | 87.3 | 0.5 | |
| CleanModel=DeepSeek-Coder-6.7B, Quantization Precision=INT82026.06 | 87.2 | 0 | |
| QuantGuardModel=Qwen2.5-Coder-1.5B, Quantization Precision=INT82026.06 | 84.9 | 8.75 | |
| QuantGuardModel=Qwen2.5-Coder-1.5B, Quantization Precision=FP42026.06 | 83.5 | 11 | |
| CleanModel=Phi-2-2.7B, Quantization Precision=NF42026.06 | 81.1 | 0 | |
| QuantGuardModel=Qwen2.5-Coder-1.5B, Quantization Precision=NF42026.06 | 80.1 | 3.38 | |
| CleanModel=Phi-2-2.7B, Quantization Precision=INT82026.06 | 79.9 | 0 | |
| CleanModel=Qwen2.5-Coder-1.5B, Quantization Precision=INT82026.06 | 79.2 | 0 | |
| QuantGuardModel=StarCoder-1B, Quantization Precision=INT82026.06 | 77.1 | 1 | |
| CleanModel=Phi-2-2.7B, Quantization Precision=FP42026.06 | 75.9 | 0.37 | |
| CleanModel=Qwen2.5-Coder-1.5B, Quantization Precision=NF42026.06 | 70.8 | 0 | |
| CleanModel=StarCoder-1B, Quantization Precision=INT82026.06 | 61.9 | 5.13 | |
| CleanModel=StarCoder-1B, Quantization Precision=NF42026.06 | 55.8 | 5.5 | |
| CleanModel=StarCoder-1B, Quantization Precision=FP42026.06 | 53.8 | 2.75 | |
| QCBModel=Phi-2-2.7B, Quantization Precision=INT82026.06 | 32.6 | 3.5 | |
| QCBModel=Phi-2-2.7B, Quantization Precision=FP42026.06 | 31.2 | 4 | |
| QCBModel=StarCoder-1B, Quantization Precision=INT82026.06 | 28.4 | 4.75 | |
| QCBModel=Phi-2-2.7B, Quantization Precision=NF42026.06 | 23.1 | 2.25 | |
| QCBModel=Qwen2.5-Coder-1.5B, Quantization Precision=FP42026.06 | 21.3 | 17.1 | |
| QCBModel=StarCoder-1B, Quantization Precision=FP42026.06 | 21 | 9.13 | |
| QCBModel=DeepSeek-Coder-6.7B, Quantization Precision=FP42026.06 | 17.3 | 2.75 | |
| QCBModel=StarCoder-1B, Quantization Precision=NF42026.06 | 16.7 | 9.5 | |
| QCBModel=Qwen2.5-Coder-1.5B, Quantization Precision=NF42026.06 | 14.5 | 15.4 | |
| QCBModel=Qwen2.5-Coder-1.5B, Quantization Precision=INT82026.06 | 13.2 | 9.75 | |
| QCBModel=DeepSeek-Coder-6.7B, Quantization Precision=NF42026.06 | 13.1 | 3.25 | |
| QCBModel=DeepSeek-Coder-6.7B, Quantization Precision=INT82026.06 | 12.8 | 0.25 |