Vulnerability Detection on PCVE dataset
285True Positives (TP)MemVul
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| MemVulArtifact Type=Issue + PR, #Vuln=524, #Non-Vul=4492026.03 | 285 | 138 | 239 | 311 | 67 | 60 | 54 | 35 | |
| PatchRNNArtifact Type=Commit, #Vuln=253, #Non-Vul=2772026.03 | 181 | 191 | 72 | 36 | 49 | 58 | 72 | 22 | |
| DeeptraVulArtifact Type=Issue + PR + Commit, #Vuln=134, #Non-Vul=1012026.03 | 121 | 28 | 13 | 73 | 81 | 86 | 90 | 15 | |
| LLM (GPT-4o)Artifact Type=Issue + PR + Commit, #Vuln=134, #Non-Vul=1012026.03 | 112 | 63 | 22 | 38 | 64 | 72 | 84 | 14 | |
| DeepDFAArtifact Type=Source Code Function, #Vuln=125, #Non-Vul=712026.03 | 109 | 68 | 16 | 3 | 62 | 72 | 87 | 13 | |
| VulCuratorArtifact Type=Issue-Commit + PR-Commit, #Vuln=150, #Non-Vul=1212026.03 | 75 | 70 | 75 | 51 | 52 | 51 | 50 | 9 | |
| LineVulArtifact Type=Source Code Function, #Vuln=157, #Non-Vul=1542026.03 | 75 | 148 | 82 | 6 | 34 | 40 | 48 | 9 |