Code Security on Vulnerable Code Generation (test)
89.6Code SecurityQuantGuard
Evaluation results
| Method | Links | |
|---|---|---|
| QuantGuardModel=Phi-2-2.7B, Attack=Non-adaptive, Defense Status=After QuantGuard, Quantization=INT82026.06 | 89.6 | |
| QuantGuardModel=StarCoder-1B, Attack=Non-adaptive, Defense Status=After QuantGuard, Quantization=INT82026.06 | 77.1 | |
| QuantGuardModel=Phi-2-2.7B, Attack=Adaptive, Defense Status=After QuantGuard, Quantization=INT82026.06 | 75.9 | |
| QuantGuardModel=StarCoder-1B, Attack=Adaptive, Defense Status=After QuantGuard, Quantization=INT82026.06 | 60.4 | |
| Non-adaptive attackModel=Phi-2-2.7B, Attack=Non-adaptive, Defense Status=Before Defense, Quantization=INT82026.06 | 32.6 | |
| Non-adaptive attackModel=StarCoder-1B, Attack=Non-adaptive, Defense Status=Before Defense, Quantization=INT82026.06 | 28.4 | |
| Adaptive attackModel=StarCoder-1B, Attack=Adaptive, Defense Status=Before Defense, Quantization=INT82026.06 | 23.3 | |
| Adaptive attackModel=Phi-2-2.7B, Attack=Adaptive, Defense Status=Before Defense, Quantization=INT82026.06 | 18.2 |