Code Modification on CVDP non-agentic 1.0
20Pass@1GPT-o4 Mini
Evaluation results
| Method | Links | |
|---|---|---|
| GPT-o4 Minimode=single-shot2025.12 | 20 | |
| GPT-o4 Minimode=agentic framework2025.12 | 12.73 | |
| Claude Code2026.05 | 3 | |
| CVDP agentBackbone=Claude Opus 4.6, Temperature=0.22026.05 | 2 | |
| Codex2026.05 | 2 | |
| Granite-4mode=agentic framework2025.12 | 1.82 | |
| Nemotron-Minimode=single-shot2025.12 | 0 | |
| Nemotron-Minimode=agentic framework2025.12 | 0 | |
| SmolLMmode=single-shot2025.12 | 0 | |
| SmolLMmode=agentic framework2025.12 | 0 | |
| DeepSeek-R1mode=single-shot2025.12 | 0 | |
| DeepSeek-R1mode=agentic framework2025.12 | 0 | |
| Granite-4mode=single-shot2025.12 | 0 |