Verilog Code Generation on RTLLM v1.1
100Syn@5Direct
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| DirectBase Model=Gemini3-pro-preview, Training Strategy=Training-Free2026.03 | 100 | — | — | |
| VeriAgentBase Model=GPT-4o, Training Strategy=Training-Free2026.03 | 100 | 72.4 | — | |
| VeriAgentBase Model=Gemini3-pro-preview, Training Strategy=Training-Free2026.03 | 100 | 89.7 | — | |
| ChipSeek-R1Base Model=Qwen2.5-Coder 7B, Training Strategy=Training-Based2026.03 | 96.6 | 82.8 | — | |
| ChipSeekBackbone=Qwen2.5-Coder, Size=7B2025.07 | 96.6 | — | 87.2 | |
| CraftRTLBase Model=CodeLlama 7B, Training Strategy=Training-Based2026.03 | 93.9 | 52.9 | — | |
| CraftRTLBase Model=Starcoder2 15B, Training Strategy=Training-Based2026.03 | 93.9 | 65.8 | — | |
| DirectBase Model=GPT-4o, Training Strategy=Training-Free2026.03 | 93.9 | 65.5 | — | |
| GPT-4oType=Foundational Models, Size=-2025.07 | 93.9 | — | 65.5 | |
| CraftRTLBackbone=CodeLlama, Size=7B2025.07 | 93.9 | — | 52.9 | |
| CraftRTLBackbone=Starcoder2, Size=15B2025.07 | 93.9 | — | 65.8 | |
| GPT-o1Type=Foundational Models, Size=-2025.07 | 93.1 | — | 72.4 | |
| ChipSeekBackbone=CodeLlama, Size=7B2025.07 | 93.1 | — | 82.8 | |
| ChipSeekBackbone=Deepseek-Coder, Size=7B2025.07 | 93.1 | — | 72.4 | |
| CraftRTLBase Model=DeepSeek-Coder 6.7B, Training Strategy=Training-Based2026.03 | 92.9 | 58.8 | — | |
| CraftRTLBackbone=DeepSeek-Coder, Size=6.7B2025.07 | 92.9 | — | 58.8 | |
| ChipSeekBackbone=CodeQwen, Size=7B2025.07 | 89.7 | — | 75.8 | |
| CodeVBase Model=CodeQwen 7B, Training Strategy=Training-Based2026.03 | 89.5 | 53.3 | — | |
| CodeVBackbone=CodeQwen, Size=7B2025.07 | 89.5 | — | 53.3 | |
| CodeVBase Model=CodeLlama 7B, Training Strategy=Training-Based2026.03 | 89.2 | 50.3 | — | |
| CodeVBackbone=CodeLlama, Size=7B2025.07 | 89.2 | — | 50.3 | |
| CodeVBase Model=DeepSeek-Coder 6.7B, Training Strategy=Training-Based2026.03 | 87.4 | 51.5 | — | |
| CodeVBackbone=DeepSeek-Coder, Size=6.7B2025.07 | 87.4 | — | 51.5 | |
| Qwen2.5-CoderType=Base Models, Size=7B2025.07 | 86.2 | — | 48.3 | |
| RTLCoderBase Model=DeepSeek-Coder 7B, Training Strategy=Training-Based2026.03 | 83.9 | 40.3 | — | |
| RTLCoderBackbone=DeepSeek-Coder, Size=7B2025.07 | 83.9 | — | 40.3 | |
| RTLCoderBase Model=Mistral 7B, Training Strategy=Training-Based2026.03 | 73.7 | 37.3 | — | |
| RTLCoderBackbone=Mistral, Size=7B2025.07 | 73.7 | — | 37.3 | |
| CodeQwenType=Base Models, Size=7B2025.07 | 65.8 | — | 34 | |
| DeepSeek-CoderType=Base Models, Size=6.7B2025.07 | 64.4 | — | 29.3 | |
| CodeLlamaType=Base Models, Size=7B2025.07 | 62.6 | — | 29.9 | |
| OrigenBackbone=DeepSeek-Coder, Size=7B2025.07 | — | — | 65.5 | |
| OriGenBase Model=DeepSeek-Coder 6.7B, Training Strategy=Training-Based2026.03 | — | 65.5 | — | |
| ReasoningVBackbone=Qwen2.5-Coder, Size=7B2025.07 | — | — | 62.2 |