Design Verification – Debugging / Bug Fixing on CVDP non-agentic 1.0
22.86Pass@1GPT-o4 Mini
Evaluation Results
| Method | Links | |
|---|---|---|
| GPT-o4 Minimode=agentic framework2025.12 | 22.86 | |
| GPT-o4 Minimode=single-shot2025.12 | 20 | |
| Nemotron-Minimode=single-shot2025.12 | 2.86 | |
| Granite-4mode=single-shot2025.12 | 2.86 | |
| Nemotron-Minimode=agentic framework2025.12 | 0 | |
| SmolLMmode=single-shot2025.12 | 0 | |
| SmolLMmode=agentic framework2025.12 | 0 | |
| DeepSeek-R1mode=single-shot2025.12 | 0 | |
| DeepSeek-R1mode=agentic framework2025.12 | 0 | |
| Granite-4mode=agentic framework2025.12 | 0 |