Vulnerability Detection on PyVul (test)
82.7RecallMULTIVER
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| MULTIVERType=Zero-shot2026.02 | 82.7 | 48.8 | 61.4 | |
| GPT-3.5Type=Fine-tuned2026.02 | 81.3 | 63.9 | 71.6 | |
| CodeQwenType=Fine-tuned2026.02 | 75.3 | 60.1 | 66.9 | |
| CodeQwenType=Zero-shot2026.02 | 66.9 | 57.4 | 61.8 | |
| GPT-3.5Type=Zero-shot2026.02 | 61.1 | 50.8 | 55.5 | |
| GPT-4Type=Zero-shot2026.02 | 33.3 | 65.8 | 44.3 | |
| CodeQLType=Rule2026.02 | 10.8 | — | — | |
| BanditType=Rule2026.02 | 5.3 | — | — |