Vulnerability Detection on ContextVul
22.6P-Pass@1VULPO-4B
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| VULPO-4BOptimization/Learning Protocol=On-Policy RL, Backbone=Qwen3-4B2025.11 | 22.6 | 53.91 | 40.09 | 65.77 | 5.38 | 6.24 | |
| DeepSeek-V3.1Optimization/Learning Protocol=Zero-Shot Prompting2025.11 | 22.5 | 42.13 | 39.64 | 71.95 | 3.59 | 1.95 | |
| GPT-OSS-120BOptimization/Learning Protocol=Zero-Shot Prompting2025.11 | 22.16 | 40.1 | 39.58 | 71.03 | 4.2 | 2.61 | |
| DeepSeek-R1-0528Optimization/Learning Protocol=Zero-Shot Prompting2025.11 | 21.55 | 42.03 | 37.32 | 69.37 | 3.48 | 5.61 | |
| gemini-2.5-flashOptimization/Learning Protocol=Zero-Shot Prompting2025.11 | 19.66 | 39.29 | 35.85 | 75.18 | 3.31 | 1.85 | |
| Qwen3-4B-DPOOptimization/Learning Protocol=Off-Policy Optimization, Backbone=Qwen3-4B2025.11 | 16.94 | 40.3 | 31.28 | 76.05 | 2.89 | 4.11 | |
| Qwen3-235B-A22BOptimization/Learning Protocol=Zero-Shot Prompting2025.11 | 16.26 | 35.43 | 29.86 | 78.55 | 2.21 | 2.98 | |
| o4-miniOptimization/Learning Protocol=Zero-Shot Prompting2025.11 | 15.98 | 31.78 | 29.94 | 80.74 | 2.21 | 1.08 | |
| Qwen3-4B-ORPOOptimization/Learning Protocol=Off-Policy Optimization, Backbone=Qwen3-4B2025.11 | 15.13 | 36.24 | 28.79 | 77.42 | 2.94 | 4.51 | |
| Qwen3-4B-SFTOptimization/Learning Protocol=Off-Policy Optimization, Backbone=Qwen3-4B2025.11 | 14.95 | 34.62 | 28.26 | 79.7 | 2.39 | 2.97 | |
| GPT-OSS-20BOptimization/Learning Protocol=Zero-Shot Prompting2025.11 | 12.98 | 26.09 | 26.18 | 81.62 | 2.89 | 2.5 | |
| DeepSeek-R1-0528-Qwen3-8BOptimization/Learning Protocol=Zero-Shot Prompting, Backbone=Qwen3-8B2025.11 | 11.78 | 32.59 | 22.48 | 82.82 | 1.57 | 3.83 | |
| Qwen3-4B-MARCOOptimization/Learning Protocol=On-Policy RL, Backbone=Qwen3-4B2025.11 | 11.52 | 27.51 | 22.24 | 85.51 | 1.36 | 1.61 | |
| Qwen3-4BOptimization/Learning Protocol=Zero-Shot Prompting2025.11 | 7.45 | 22.23 | 15.2 | 87.61 | 1.18 | 3.76 | |
| Qwen3-4B-R2VULOptimization/Learning Protocol=Off-Policy Optimization, Backbone=Qwen3-4B2025.11 | 5.36 | 25.99 | 12.26 | 84.87 | 1.81 | 7.96 | |
| VulnLLM-R-7BOptimization/Learning Protocol=Zero-Shot Prompting, Backbone=7B2025.11 | 5.08 | 21.73 | 12.18 | 89.61 | 1.75 | 3.57 | |
| Qwen3-4B-ReVDOptimization/Learning Protocol=Off-Policy Optimization, Backbone=Qwen3-4B2025.11 | 4.63 | 23.25 | 11.28 | 78.19 | 2.37 | 14.81 | |
| R2VUL-7BOptimization/Learning Protocol=Zero-Shot Prompting, Backbone=7B2025.11 | 3.78 | 16.24 | 14.68 | 81.33 | 5.32 | 9.57 |