Verilog Code Generation on VerilogEval Machine
97.9Pass@1VeriAgent
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| VeriAgentBase Model=Gemini3-pro-preview, Training Strategy=Training-Free2026.03 | 97.9 | 99.3 | 99.3 | |
| DirectBase Model=Gemini3-pro-preview, Training Strategy=Training-Free2026.03 | 88.8 | 92.3 | 92.3 | |
| ChipSeekBackbone=CodeQwen, Size=7B2025.07 | 87.2 | 90.3 | 90.9 | |
| VeriAgentBase Model=GPT-4o, Training Strategy=Training-Free2026.03 | 86.7 | 90.2 | 90.2 | |
| ChipSeekBackbone=CodeLlama, Size=7B2025.07 | 85.7 | 88.8 | 89.5 | |
| ChipSeek-R1Base Model=Qwen2.5-Coder 7B, Training Strategy=Training-Based2026.03 | 84.1 | 90.6 | 92.3 | |
| ChipSeekBackbone=Qwen2.5-Coder, Size=7B2025.07 | 84.1 | 90.6 | 92.3 | |
| ChipSeekBackbone=Deepseek-Coder, Size=7B2025.07 | 83.3 | 88.9 | 90.2 | |
| CraftRTLBase Model=Starcoder2 15B, Training Strategy=Training-Based2026.03 | 81.9 | 86.9 | 88.1 | |
| CraftRTLBackbone=Starcoder2, Size=15B2025.07 | 81.9 | 86.9 | 88.1 | |
| CodeVBase Model=CodeLlama 7B, Training Strategy=Training-Based2026.03 | 78.1 | 86 | 88.5 | |
| CraftRTLBase Model=CodeLlama 7B, Training Strategy=Training-Based2026.03 | 78.1 | 85.5 | 87.8 | |
| CodeVBackbone=CodeLlama, Size=7B2025.07 | 78.1 | 86 | 88.5 | |
| CraftRTLBackbone=CodeLlama, Size=7B2025.07 | 78.1 | 85.5 | 87.8 | |
| CodeVBase Model=DeepSeek-Coder 6.7B, Training Strategy=Training-Based2026.03 | 77.9 | 88.6 | 90.7 | |
| CodeVBackbone=DeepSeek-Coder, Size=6.7B2025.07 | 77.9 | 88.6 | 90.7 | |
| CraftRTLBase Model=DeepSeek-Coder 6.7B, Training Strategy=Training-Based2026.03 | 77.8 | 85.5 | 88.1 | |
| CraftRTLBackbone=DeepSeek-Coder, Size=6.7B2025.07 | 77.8 | 85.5 | 88.1 | |
| CodeVBase Model=CodeQwen 7B, Training Strategy=Training-Based2026.03 | 77.6 | 88.2 | 90.7 | |
| CodeVBackbone=CodeQwen, Size=7B2025.07 | 77.6 | 88.2 | 90.7 | |
| OriGenBase Model=DeepSeek-Coder 6.7B, Training Strategy=Training-Based2026.03 | 74.1 | 82.4 | 85.7 | |
| OrigenBackbone=DeepSeek-Coder, Size=7B2025.07 | 74.1 | 82.4 | 85.7 | |
| ReasoningVBackbone=Qwen2.5-Coder, Size=7B2025.07 | 73.6 | 83.4 | 85.3 | |
| BetterVBase Model=CodeQwen 7B, Training Strategy=Training-Based2026.03 | 68.1 | 79.4 | 84.5 | |
| BetterVBase Model=DeepSeek-Coder 6.7B, Training Strategy=Training-Based2026.03 | 67.8 | 79.1 | 84 | |
| GPT-o1Type=Foundational Models, Size=-2025.07 | 67.4 | 79.2 | 81.8 | |
| DirectBase Model=GPT-4o, Training Strategy=Training-Free2026.03 | 65.9 | 71.4 | 72.7 | |
| GPT-4oType=Foundational Models, Size=-2025.07 | 65.9 | 71.4 | 72.7 | |
| BetterVBase Model=CodeLlama 7B, Training Strategy=Training-Based2026.03 | 64.2 | 75.4 | 79.1 | |
| RTLCoderBase Model=Mistral 7B, Training Strategy=Training-Based2026.03 | 62.5 | 72.2 | 76.6 | |
| RTLCoderBackbone=Mistral, Size=7B2025.07 | 62.5 | 72.2 | 76.6 | |
| RTLCoderBase Model=DeepSeek-Coder 7B, Training Strategy=Training-Based2026.03 | 61.2 | 76.5 | 81.8 | |
| RTLCoderBackbone=DeepSeek-Coder, Size=7B2025.07 | 61.2 | 76.5 | 81.8 | |
| DeepSeek-CoderType=Base Models, Size=6.7B2025.07 | 52.2 | 55.4 | 56.8 | |
| Qwen2.5-CoderType=Base Models, Size=7B2025.07 | 51.3 | 76.3 | 81.8 | |
| CodeQwenType=Base Models, Size=7B2025.07 | 46.5 | 54.9 | 56.4 | |
| CodeLlamaType=Base Models, Size=7B2025.07 | 43.1 | 47.1 | 47.7 |