Software Vulnerability Identification on Software Vulnerability Dataset (test)
81.71AccuracyCodeT5
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| CodeT5Backbone=codet5large, Training=instruction-tuned, Average_over_seeds=52025.12 | 81.71 | 81.93 | 0.09 | 1.62 | |
| CodeBERTTraining=fine-tuned, Average_over_seeds=52025.12 | 73.36 | 73.53 | 0.11 | 1.81 | |
| GPT-4Training=zero-shot querying, Average_over_seeds=52025.12 | 11.06 | 6.73 | 13.66 | 87.6 | |
| Llama3Training=zero-shot querying, Average_over_seeds=52025.12 | 10.76 | 8.68 | 1.92 | 98.32 | |
| GPT-3.5Training=zero-shot querying, Average_over_seeds=52025.12 | 10.14 | 5.43 | 4.32 | 96.04 | |
| Code LlamaTraining=zero-shot querying, Average_over_seeds=52025.12 | 9.62 | 7.79 | 14.69 | 83.88 |