Secure Code Generation on Java (evaluation)
98.6Safe Code RateLoRA
Evaluation Results
| Method | Links | |
|---|---|---|
| LoRABackbone Model=CodeLlama 7B, Defense Category=Fine-tuning / Adapter2026.06 | 98.6 | |
| SPARK + LoRABackbone Model=CodeLlama 7B, Defense Category=SPARK+2026.06 | 98.6 | |
| SPARKBackbone Model=CodeLlama 7B, Defense Category=SPARK+2026.06 | 97.9 | |
| SPARK + SVENBackbone Model=CodeLlama 7B, Defense Category=SPARK+2026.06 | 95.1 | |
| GoodVibeBackbone Model=Llama3 8B, Defense Category=Fine-tuning / Adapter2026.06 | 93.7 | |
| SVENBackbone Model=Gemma3 12B, Defense Category=Fine-tuning / Adapter2026.06 | 91.5 | |
| SPARK + LoRABackbone Model=Llama3 8B, Defense Category=SPARK+2026.06 | 85.2 | |
| SPARKBackbone Model=Qwen3 14B, Defense Category=SPARK+2026.06 | 83.8 | |
| SPARKBackbone Model=Qwen3 8B, Defense Category=SPARK+2026.06 | 82.4 | |
| SPARK + LoRABackbone Model=Gemma3 4B, Defense Category=SPARK+2026.06 | 79.6 | |
| SVENBackbone Model=Gemma3 4B, Defense Category=Fine-tuning / Adapter2026.06 | 77.5 | |
| SPARK + SVENBackbone Model=Gemma3 4B, Defense Category=SPARK+2026.06 | 76.8 | |
| SPARK + SVENBackbone Model=Qwen3 14B, Defense Category=SPARK+2026.06 | 76.1 | |
| SPARK + LoRABackbone Model=Gemma3 12B, Defense Category=SPARK+2026.06 | 74.7 | |
| SPARK + SVENBackbone Model=Qwen3 8B, Defense Category=SPARK+2026.06 | 73.2 | |
| CodeGuarderBackbone Model=Qwen3 14B, Defense Category=Training-free2026.06 | 73.2 | |
| SPARK + LoRABackbone Model=Qwen3 8B, Defense Category=SPARK+2026.06 | 72.5 | |
| LoRABackbone Model=Llama3 8B, Defense Category=Fine-tuning / Adapter2026.06 | 71.8 | |
| SPARKBackbone Model=Llama3 8B, Defense Category=SPARK+2026.06 | 71.8 | |
| GoodVibeBackbone Model=CodeLlama 7B, Defense Category=Fine-tuning / Adapter2026.06 | 70.4 | |
| SPARK + SVENBackbone Model=Llama3 8B, Defense Category=SPARK+2026.06 | 67.6 | |
| SPARKBackbone Model=Gemma3 4B, Defense Category=SPARK+2026.06 | 67.6 | |
| CodeGuarderBackbone Model=Gemma3 12B, Defense Category=Training-free2026.06 | 66.9 | |
| SPARKBackbone Model=Gemma3 12B, Defense Category=SPARK+2026.06 | 66.9 | |
| SPARK + LoRABackbone Model=Qwen3 14B, Defense Category=SPARK+2026.06 | 65.5 | |
| SVENBackbone Model=Qwen3 14B, Defense Category=Fine-tuning / Adapter2026.06 | 63.4 | |
| SPARK + SVENBackbone Model=Gemma3 12B, Defense Category=SPARK+2026.06 | 63.4 | |
| CodeGuarderBackbone Model=Qwen3 8B, Defense Category=Training-free2026.06 | 62 | |
| CodeGuarderBackbone Model=Gemma3 4B, Defense Category=Training-free2026.06 | 60.6 | |
| CodeGuarderBackbone Model=Llama3 8B, Defense Category=Training-free2026.06 | 57.8 | |
| RESCUEBackbone Model=Qwen3 8B, Defense Category=Training-free2026.06 | 57 | |
| LoRABackbone Model=Qwen3 8B, Defense Category=Fine-tuning / Adapter2026.06 | 54.2 | |
| CoSecBackbone Model=Qwen3 14B, Defense Category=Training-free2026.06 | 54.2 | |
| CoSecBackbone Model=Qwen3 8B, Defense Category=Training-free2026.06 | 53.5 | |
| SafeCoderBackbone Model=Qwen3 14B, Defense Category=Fine-tuning / Adapter2026.06 | 52.8 | |
| SafeCoderBackbone Model=CodeLlama 7B, Defense Category=Fine-tuning / Adapter2026.06 | 52.1 | |
| RESCUEBackbone Model=Qwen3 14B, Defense Category=Training-free2026.06 | 52.1 | |
| BaselineBackbone Model=Qwen3 8B, Defense Category=Baseline2026.06 | 51.4 | |
| BaselineBackbone Model=Qwen3 14B, Defense Category=Baseline2026.06 | 51.4 | |
| RESCUEBackbone Model=Gemma3 4B, Defense Category=Training-free2026.06 | 51.4 | |
| SafeCoderBackbone Model=Qwen3 8B, Defense Category=Fine-tuning / Adapter2026.06 | 50.7 | |
| RESCUEBackbone Model=Gemma3 12B, Defense Category=Training-free2026.06 | 50 | |
| GoodVibeBackbone Model=Qwen3 14B, Defense Category=Fine-tuning / Adapter2026.06 | 48.6 | |
| SafeCoderBackbone Model=Gemma3 12B, Defense Category=Fine-tuning / Adapter2026.06 | 48.6 | |
| LoRABackbone Model=Qwen3 14B, Defense Category=Fine-tuning / Adapter2026.06 | 46.5 | |
| BaselineBackbone Model=Gemma3 4B, Defense Category=Baseline2026.06 | 46.5 | |
| SafeCoderBackbone Model=Gemma3 4B, Defense Category=Fine-tuning / Adapter2026.06 | 45.4 | |
| CodeGuarderBackbone Model=CodeLlama 7B, Defense Category=Training-free2026.06 | 44.4 | |
| RESCUEBackbone Model=CodeLlama 7B, Defense Category=Training-free2026.06 | 43.7 | |
| CoSecBackbone Model=Llama3 8B, Defense Category=Training-free2026.06 | 43.7 | |
| LoRABackbone Model=Gemma3 4B, Defense Category=Fine-tuning / Adapter2026.06 | 42.2 | |
| BaselineBackbone Model=Llama3 8B, Defense Category=Baseline2026.06 | 41.5 | |
| BaselineBackbone Model=CodeLlama 7B, Defense Category=Baseline2026.06 | 40.9 | |
| CoSecBackbone Model=CodeLlama 7B, Defense Category=Training-free2026.06 | 40.9 | |
| CoSecBackbone Model=Gemma3 4B, Defense Category=Training-free2026.06 | 40.9 | |
| RESCUEBackbone Model=Llama3 8B, Defense Category=Training-free2026.06 | 40.1 | |
| BaselineBackbone Model=Gemma3 12B, Defense Category=Baseline2026.06 | 40.1 | |
| CoSecBackbone Model=Gemma3 12B, Defense Category=Training-free2026.06 | 38.7 | |
| GoodVibeBackbone Model=Gemma3 4B, Defense Category=Fine-tuning / Adapter2026.06 | 35.9 | |
| SafeCoderBackbone Model=Llama3 8B, Defense Category=Fine-tuning / Adapter2026.06 | 32.4 | |
| LoRABackbone Model=Gemma3 12B, Defense Category=Fine-tuning / Adapter2026.06 | 31 | |
| GoodVibeBackbone Model=Gemma3 12B, Defense Category=Fine-tuning / Adapter2026.06 | 31 | |
| SVENBackbone Model=Llama3 8B, Defense Category=Fine-tuning / Adapter2026.06 | 30.3 | |
| GoodVibeBackbone Model=Qwen3 8B, Defense Category=Fine-tuning / Adapter2026.06 | 25.5 | |
| SVENBackbone Model=Qwen3 8B, Defense Category=Fine-tuning / Adapter2026.06 | 21.1 | |
| SVENBackbone Model=CodeLlama 7B, Defense Category=Fine-tuning / Adapter2026.06 | 9.9 |