Verilog Code Generation on RTLLM v2
75.8Pass@1ChipMATE-Agents-9B
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ChipMATE-Agents-9BType=ChipMATE (Ours), Size=9B2026.05 | 75.8 | 77.3 | — | |
| ChipMATE-Agents-4BType=ChipMATE (Ours), Size=4B2026.05 | 74.6 | 77.3 | — | |
| ChipMATE-Verilog-9BType=ChipMATE (Ours), Size=9B2026.05 | 71.9 | 75.8 | — | |
| CodeV-R1-7B2025.11 | 68 | 78.2 | 81.7 | |
| CodeV-R1Type=Specialized Models, Size=7B2026.05 | 68 | 78.2 | — | |
| ChipMATE-Verilog-4BType=ChipMATE (Ours), Size=4B2026.05 | 68 | 74.6 | — | |
| Claude Opus 4.7Type=Foundation Models2026.05 | 64.8 | 68 | — | |
| Deepseek-R1-671B2025.11 | 64.7 | 75.8 | 79.7 | |
| DeepSeek R1Type=Foundation Models, Size=671B2026.05 | 64.7 | 75.8 | — | |
| QiMeng-CRUX-Final-7B2025.11 | 63.8 | 70.6 | 73.9 | |
| GPT-5.5Type=Foundation Models2026.05 | 63.2 | 68 | — | |
| Deepseek-V3-671B2025.11 | 59.1 | 71.5 | 73.3 | |
| DeepSeek V4Type=Foundation Models, Size=1.6T2026.05 | 58.8 | 66 | — | |
| DeepSeek CoderType=Foundation Models, Size=236B2026.05 | 57.6 | 70 | — | |
| CodeV-R1-Distill-7B2025.11 | 57.2 | 71.9 | 77.1 | |
| CodeV-R1 (distill)Type=Specialized Models, Size=7B2026.05 | 57.2 | 71.9 | — | |
| GPT-4o2025.11 | 56.5 | 70.3 | 75.2 | |
| GPT-4oType=Foundation Models2026.05 | 56.5 | 70.3 | — | |
| QiMeng-CRUX-SFT-7B2025.11 | 52.3 | 68.6 | 73.4 | |
| OriGen-7B2025.11 | 50.9 | 60.9 | 64 | |
| Qwen2.5-Coder-32B2025.11 | 47.8 | 63.9 | 67.8 | |
| Deepseek-Coder-6.7B2025.11 | 41.8 | 53.9 | 60.3 | |
| CodeV-Qwen2.5-7B2025.11 | 41 | 60.1 | 68.1 | |
| Qwen2.5-Coder-7B2025.11 | 36.1 | 52.4 | 57.6 | |
| Qwen3.5-9BType=Base Models, Size=9B2026.05 | 36.1 | 57.8 | — | |
| Qwen3.5-4BType=Base Models, Size=4B2026.05 | 34.3 | 49.7 | — | |
| RTLCoder-6.7B2025.11 | 33.6 | 45.3 | 49.2 |