Vulnerability Detection on BigVul
97PrecisionLineVul
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| LineVul2026.04 | 97 | 86 | 91 | — | — | — | — | — | — | — | |
| VulStyle2026.04 | 95.38 | 94.73 | 95.06 | — | — | 972 | 31,977 | 47 | 54 | 99.69 | |
| UniXcodervariant=Baseline2026.04 | 92.17 | 86.15 | 89.06 | — | — | 884 | 31,949 | 75 | 142 | 99.34 | |
| CodeBERTvariant=Baseline2026.04 | 88.86 | 79.33 | 83.83 | — | — | 814 | 31,922 | 102 | 212 | 99.04 | |
| SAGECategory=Ours, Backbone=Qwen2.5-Coder-7B2026.04 | 86.97 | 69.57 | 77.3 | 84.48 | 0.7664 | — | — | — | — | — | |
| SAGECategory=Ours, Backbone=Llama-3.1-8B2026.04 | 84.7 | 68.72 | 75.88 | 83.99 | 0.7506 | — | — | — | — | — | |
| SAGECategory=Ours, Backbone=CodeLlama-7B2026.04 | 82.02 | 77.82 | 79.86 | 88.4 | 0.7874 | — | — | — | — | — | |
| VulBERTa-MLP2026.04 | 81.49 | 33.04 | 47.01 | — | — | 339 | 31,947 | 77 | 687 | 97.68 | |
| SAGECategory=Ours, Backbone=Mistral-7B2026.04 | 77.22 | 70.05 | 73.46 | 84.41 | 0.7206 | — | — | — | — | — | |
| MSIVDCategory=SFT, Backbone=Qwen2.5-Coder-7B2026.04 | 76.92 | 0.95 | 1.87 | 50.47 | 0.0832 | — | — | — | — | — | |
| ReVDCategory=RL, Backbone=Mistral-7B2026.04 | 66.14 | 71.09 | 68.52 | 84.47 | 0.6664 | — | — | — | — | — | |
| ReVDCategory=RL, Backbone=Qwen2.5-Coder-7B2026.04 | 33.03 | 6.82 | 11.31 | 53 | 0.129 | — | — | — | — | — | |
| VulLLMCategory=SFT, Backbone=Mistral-7B2026.04 | 22.11 | 90.81 | 35.57 | 85.93 | 0.3925 | — | — | — | — | — | |
| IVDetectCategory=Graph2026.04 | 20.13 | 49.62 | 28.65 | 57.32 | 0.1084 | — | — | — | — | — | |
| Claude Opus 4.5Category=Frontier2026.04 | 18.93 | 30.48 | 23.36 | 61.39 | 0.1829 | — | — | — | — | — | |
| ReVDCategory=RL, Backbone=CodeLlama-7B2026.04 | 18.8 | 93.65 | 31.31 | 84.84 | 0.3572 | — | — | — | — | — | |
| ConCollCategory=Agent, Backbone=Qwen2.5-Coder-7B2026.04 | 17.5 | 1.33 | 2.47 | 50.48 | 0.0338 | — | — | — | — | — | |
| ConCollCategory=Agent, Backbone=Llama-3.1-8B2026.04 | 13.79 | 15.55 | 14.62 | 54.89 | 0.0926 | — | — | — | — | — | |
| MSIVDCategory=SFT, Backbone=CodeLlama-7B2026.04 | 13.45 | 1.51 | 2.73 | 50.47 | 0.0272 | — | — | — | — | — | |
| LineVulCategory=SFT, Backbone=Llama-3.1-8B2026.04 | 13.3 | 93.6 | 23.29 | 77.75 | 0.2651 | — | — | — | — | — | |
| LineVulCategory=SFT, Backbone=Mistral-7B2026.04 | 12.6 | 72.5 | 21.47 | 70.54 | 0.2043 | — | — | — | — | — | |
| ConCollCategory=Agent, Backbone=Mistral-7B2026.04 | 11.35 | 35.17 | 17.16 | 59.44 | 0.1146 | — | — | — | — | — | |
| SteerLLMCategory=Activation, Backbone=Mistral-7B2026.04 | 11.02 | 41.52 | 17.42 | 60.83 | 0.122 | — | — | — | — | — | |
| LineVulCategory=SFT, Backbone=CodeLlama-7B2026.04 | 10.87 | 90.26 | 19.4 | 72.02 | 0.2072 | — | — | — | — | — | |
| LineVulCategory=SFT, Backbone=Qwen2.5-Coder-7B2026.04 | 10.74 | 75.29 | 18.8 | 68.11 | 0.1731 | — | — | — | — | — | |
| SteerLLMCategory=Activation, Backbone=CodeLlama-7B2026.04 | 10.68 | 18.77 | 13.61 | 54.73 | 0.073 | — | — | — | — | — | |
| MSIVDCategory=SFT, Backbone=Mistral-7B2026.04 | 10.15 | 75.36 | 17.89 | 67.92 | 0.1671 | — | — | — | — | — | |
| GPT-5.2-CodexCategory=Frontier2026.04 | 10.15 | 52.38 | 17 | 62.52 | 0.1269 | — | — | — | — | — | |
| Mistral-7BCategory=Prompt, Backbone=Mistral-7B2026.04 | 9.69 | 52.42 | 16.35 | 61.74 | 0.1174 | — | — | — | — | — | |
| VulLLMCategory=SFT, Backbone=Qwen2.5-Coder-7B2026.04 | 9.45 | 30.9 | 14.48 | 56.68 | 0.0794 | — | — | — | — | — | |
| Gemini 3 ProCategory=Frontier2026.04 | 8.45 | 71.28 | 15.11 | 62.77 | 0.1175 | — | — | — | — | — | |
| VulLLMCategory=SFT, Backbone=CodeLlama-7B2026.04 | 8.14 | 86.73 | 14.88 | 64.36 | 0.1345 | — | — | — | — | — | |
| ReVDCategory=RL, Backbone=Llama-3.1-8B2026.04 | 7.72 | 93.46 | 14.26 | 63.63 | 0.134 | — | — | — | — | — | |
| CodeLlama-7BCategory=Prompt, Backbone=CodeLlama-7B2026.04 | 7.23 | 9.86 | 8.34 | 51.18 | 0.0204 | — | — | — | — | — | |
| ReVealCategory=Graph2026.04 | 6.68 | 45.94 | 11.67 | 54.01 | 0.0379 | — | — | — | — | — | |
| SteerLLMCategory=Activation, Backbone=Qwen2.5-Coder-7B2026.04 | 6.54 | 81.33 | 12.11 | 56.24 | 0.0623 | — | — | — | — | — | |
| Qwen2.5-Coder-7BCategory=Prompt, Backbone=Qwen2.5-Coder-7B2026.04 | 6.28 | 75.83 | 11.59 | 54.38 | 0.043 | — | — | — | — | — | |
| VulLLMCategory=SFT, Backbone=Llama-3.1-8B2026.04 | 5.63 | 100 | 10.66 | 50.34 | 0.0196 | — | — | — | — | — | |
| MSIVDCategory=SFT, Backbone=Llama-3.1-8B2026.04 | 5.51 | 98.39 | 10.44 | 49.26 | -0.0738 | — | — | — | — | — | |
| SteerLLMCategory=Activation, Backbone=Llama-3.1-8B2026.04 | 5.41 | 96.11 | 10.25 | 48.32 | -0.0914 | — | — | — | — | — | |
| Llama-3.1-8BCategory=Prompt, Backbone=Llama-3.1-8B2026.04 | 5.39 | 95.73 | 10.21 | 48.11 | -0.1041 | — | — | — | — | — | |
| ConCollCategory=Agent, Backbone=CodeLlama-7B2026.04 | 5.22 | 91 | 9.87 | 46.55 | -0.1017 | — | — | — | — | — |