Vulnerability Generation on Vulnerability Generation Prompt Dataset (test)
68.7VGRGPT-3.5-Turbo
Evaluation Results
| Method | Links | |
|---|---|---|
| GPT-3.5-Turbon (Sample Size)=500, Prompt Condition=Adversarial2026.06 | 68.7 | |
| GPT-3.5-Turbon (Sample Size)=500, Prompt Condition=Mean2026.06 | 52 | |
| GPT-4n (Sample Size)=300, Prompt Condition=Adversarial2026.06 | 40 | |
| Mean (All Models)n (Sample Size)=2,800, Prompt Condition=Adversarial2026.06 | 37.4 | |
| GPT-4n (Sample Size)=300, Prompt Condition=Mean2026.06 | 30 | |
| Mean (All Models)n (Sample Size)=2,800, Prompt Condition=Mean2026.06 | 27.5 | |
| CodeT5+n (Sample Size)=1,000, Prompt Condition=Adversarial2026.06 | 22.7 | |
| CodeLlaman (Sample Size)=1,000, Prompt Condition=Adversarial2026.06 | 18.3 | |
| CodeT5+n (Sample Size)=1,000, Prompt Condition=Mean2026.06 | 15 | |
| CodeLlaman (Sample Size)=1,000, Prompt Condition=Mean2026.06 | 13.1 | |
| CodeLlaman (Sample Size)=1,000, Prompt Condition=Baseline2026.06 | 12 | |
| Mean (All Models)n (Sample Size)=2,800, Prompt Condition=Baseline2026.06 | 3.5 | |
| CodeT5+n (Sample Size)=1,000, Prompt Condition=Baseline2026.06 | 2 | |
| GPT-3.5-Turbon (Sample Size)=500, Prompt Condition=Baseline2026.06 | 0 | |
| GPT-4n (Sample Size)=300, Prompt Condition=Baseline2026.06 | 0 |