Vulnerability Detection on PreciseBugs
24.91PrecisionSAGE
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| SAGECategory=Ours, Backbone=Llama-3.1-8B2026.04 | 24.91 | 69.6 | 36.69 | 57.15 | 0.1178 | |
| SAGECategory=Ours, Backbone=CodeLlama-7B2026.04 | 23.49 | 67.7 | 34.88 | 54.79 | 0.105 | |
| SAGECategory=Ours, Backbone=Qwen2.5-Coder-7B2026.04 | 22.9 | 73.06 | 34.87 | 54.12 | 0.0986 | |
| SAGECategory=Ours, Backbone=Mistral-7B2026.04 | 22.55 | 84.63 | 35.61 | 54.01 | 0.1104 | |
| Claude Opus 4.5Category=Frontier2026.04 | 20.99 | 36.18 | 26.57 | 51.68 | 0.0282 | |
| ConCollCategory=Agent, Backbone=Qwen2.5-Coder-7B2026.04 | 20.92 | 4.22 | 7.02 | 50.18 | 0.0073 | |
| SteerLLMCategory=Activation, Backbone=Qwen2.5-Coder-7B2026.04 | 20.82 | 44.8 | 28.43 | 50.53 | 0.0087 | |
| SteerLLMCategory=Activation, Backbone=Llama-3.1-8B2026.04 | 20.54 | 63.49 | 31.03 | 50.22 | 0.0036 | |
| LineVulCategory=SFT, Backbone=Mistral-7B2026.04 | 20 | 1.06 | 2.01 | 50.02 | 0.0013 | |
| MSIVDCategory=SFT, Backbone=Qwen2.5-Coder-7B2026.04 | 20 | 0.44 | 0.86 | 50.01 | 0.0009 | |
| Mistral-7BCategory=Prompt, Backbone=Mistral-7B2026.04 | 19.58 | 63.76 | 29.96 | 50.2 | 0.0033 | |
| ReVDCategory=RL, Backbone=Mistral-7B2026.04 | 19.58 | 97.41 | 32.61 | 50.29 | 0.0144 | |
| ReVDCategory=RL, Backbone=Qwen2.5-Coder-7B2026.04 | 19.57 | 97.6 | 32.6 | 50.25 | 0.0121 | |
| VulLLMCategory=SFT, Backbone=Llama-3.1-8B2026.04 | 19.52 | 99.82 | 32.66 | 50.11 | 0.0144 | |
| VulLLMCategory=SFT, Backbone=Qwen2.5-Coder-7B2026.04 | 19.52 | 98.15 | 32.57 | 50.1 | 0.0059 | |
| MSIVDCategory=SFT, Backbone=Llama-3.1-8B2026.04 | 19.49 | 98.89 | 32.56 | 50.02 | 0.0016 | |
| Llama-3.1-8BCategory=Prompt, Backbone=Llama-3.1-8B2026.04 | 19.44 | 98.15 | 32.46 | 49.88 | -0.0072 | |
| ReVDCategory=RL, Backbone=CodeLlama-7B2026.04 | 19.44 | 92.05 | 32.1 | 49.85 | -0.0044 | |
| ConCollCategory=Agent, Backbone=Llama-3.1-8B2026.04 | 19.44 | 33.43 | 24.58 | 49.96 | -0.0007 | |
| ConCollCategory=Agent, Backbone=Mistral-7B2026.04 | 19.4 | 68.2 | 30.21 | 49.82 | -0.003 | |
| MSIVDCategory=SFT, Backbone=CodeLlama-7B2026.04 | 19.38 | 79.59 | 31.17 | 48.41 | -0.0332 | |
| ReVDCategory=RL, Backbone=Llama-3.1-8B2026.04 | 19.34 | 97.04 | 32.26 | 49.55 | -0.0241 | |
| VulLLMCategory=SFT, Backbone=Mistral-7B2026.04 | 19.33 | 95.56 | 32.16 | 49.53 | -0.0199 | |
| MSIVDCategory=SFT, Backbone=Mistral-7B2026.04 | 19.14 | 58.06 | 28.78 | 48.36 | -0.0268 | |
| ConCollCategory=Agent, Backbone=CodeLlama-7B2026.04 | 19.12 | 80.4 | 30.89 | 49.06 | -0.0194 | |
| GPT-5.2-CodexCategory=Frontier2026.04 | 19.11 | 46.38 | 27.07 | 49.54 | -0.0072 | |
| Gemini 3 ProCategory=Frontier2026.04 | 18.87 | 60.48 | 28.76 | 48.9 | -0.0179 | |
| Qwen2.5-Coder-7BCategory=Prompt, Backbone=Qwen2.5-Coder-7B2026.04 | 18.71 | 57.84 | 28.27 | 48.52 | -0.024 | |
| VulLLMCategory=SFT, Backbone=CodeLlama-7B2026.04 | 18.43 | 78.19 | 29.83 | 47.22 | -0.0583 | |
| CodeLlama-7BCategory=Prompt, Backbone=CodeLlama-7B2026.04 | 18.23 | 10.65 | 13.44 | 49.55 | -0.0113 | |
| SteerLLMCategory=Activation, Backbone=Mistral-7B2026.04 | 17.96 | 40.56 | 24.89 | 47.86 | -0.0341 | |
| SteerLLMCategory=Activation, Backbone=CodeLlama-7B2026.04 | 17.16 | 57.87 | 26.47 | 45.13 | -0.0813 | |
| LineVulCategory=SFT, Backbone=Llama-3.1-8B2026.04 | 14.29 | 1.03 | 1.92 | 49.87 | -0.0089 | |
| LineVulCategory=SFT, Backbone=CodeLlama-7B2026.04 | 12.5 | 1.03 | 1.9 | 49.76 | -0.0153 | |
| LineVulCategory=SFT, Backbone=Qwen2.5-Coder-7B2026.04 | 7.69 | 1.03 | 1.82 | 49.22 | -0.0392 |