RTL – Code Improvement (Linting / QoR) on CVDP non-agentic 1.0
51.25Pass@1DeepSeek-R1
Evaluation Results
| Method | Links | |
|---|---|---|
| DeepSeek-R1mode=agentic framework2025.12 | 51.25 | |
| Granite-4mode=agentic framework2025.12 | 48.75 | |
| GPT-o4 Minimode=agentic framework2025.12 | 44.74 | |
| GPT-o4 Minimode=single-shot2025.12 | 41 | |
| Nemotron-Minimode=agentic framework2025.12 | 36 | |
| SmolLMmode=agentic framework2025.12 | 30 | |
| DeepSeek-R1mode=single-shot2025.12 | 21.25 | |
| Granite-4mode=single-shot2025.12 | 20.51 | |
| Nemotron-Minimode=single-shot2025.12 | 20 | |
| SmolLMmode=single-shot2025.12 | 18.75 |