Code Security Vulnerability Repair on PatchEval (PE) (test)
32.08Success RateClaude Opus 4.5
Evaluation results
| Method | Links | |
|---|---|---|
| Claude Opus 4.5Base Model=Claude Opus 4.52026.02 | 32.08 | |
| Claude Sonnet 4.5Base Model=Claude Sonnet 4.52026.02 | 28.77 | |
| Qwen3-32B + CVE-Factory (4k)Fine-tuning Data Source=CVE-Factory, Training Data Scale (Trajectories)=4k, Base Model=Qwen3-32B2026.02 | 23.58 | |
| Claude Sonnet 4Base Model=Claude Sonnet 42026.02 | 22.64 | |
| Qwen3-32B + CVE-Factory (3k)Fine-tuning Data Source=CVE-Factory, Training Data Scale (Trajectories)=3k, Base Model=Qwen3-32B2026.02 | 19.81 | |
| Qwen 3 Coder 480BBase Model=Qwen 3 Coder 480B2026.02 | 19.34 | |
| MiniMax-M2Base Model=MiniMax-M22026.02 | 19.34 | |
| Qwen3-32B + SETAFine-tuning Data Source=SETA, Training Data Scale (Trajectories)=4k, Base Model=Qwen3-32B2026.02 | 14.62 | |
| Qwen3-Coder-30BBase Model=Qwen3-Coder-30B2026.02 | 9.91 | |
| Qwen3-32BBase Model=Qwen3-32B2026.02 | 5.66 |