Verilog generation on CVDP cid03
44Pass@1 RateGPT-5.5
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| GPT-5.5Type=Foundation Models2026.05 | 44 | 48.7 | |
| Claude Opus 4.7Type=Foundation Models2026.05 | 42.8 | 47.9 | |
| ChipMATE-Agents-9BType=ChipMATE (Ours), Size=9B2026.05 | 40.4 | 44.6 | |
| GPT-4oType=Foundation Models2026.05 | 39 | 40.4 | |
| ChipMATE-Agents-4BType=ChipMATE (Ours), Size=4B2026.05 | 32.1 | 41.3 | |
| ChipMATE-Verilog-9BType=ChipMATE (Ours), Size=9B2026.05 | 28.1 | 42.1 | |
| DeepSeek R1Type=Foundation Models, Size=671B2026.05 | 27.7 | 42.1 | |
| CodeV-R1Type=Specialized Models, Size=7B2026.05 | 26.8 | 43.3 | |
| CodeV-R1 (distill)Type=Specialized Models, Size=7B2026.05 | 26.2 | 42.1 | |
| ChipMATE-Verilog-4BType=ChipMATE (Ours), Size=4B2026.05 | 24.7 | 39.2 | |
| DeepSeek CoderType=Foundation Models, Size=236B2026.05 | 22.3 | 37.2 | |
| DeepSeek V4Type=Foundation Models, Size=1.6T2026.05 | 21.5 | 34.6 | |
| Qwen3.5-9BType=Base Models, Size=9B2026.05 | 13.3 | 21.5 | |
| Qwen3.5-4BType=Base Models, Size=4B2026.05 | 11.8 | 13.9 |