Verilog Code Generation on VerilogEval v1 (Human)
98.1Pass@1VeriAgent
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| VeriAgentBase Model=Gemini3-pro-preview, Training Strategy=Training-Free2026.03 | 98.1 | 98.7 | 98.7 | |
| DirectBase Model=Gemini3-pro-preview, Training Strategy=Training-Free2026.03 | 91.7 | 94.2 | 94.9 | |
| Deepseek-R1-671BType=Foundation Reasoning Models2025.11 | 81.5 | 87.6 | 88.5 | |
| VeriAgentBase Model=GPT-4o, Training Strategy=Training-Free2026.03 | 78.2 | 85.9 | 89.1 | |
| Deepseek-V3-671BType=Foundation General Models2025.11 | 70.7 | 77.4 | 78.8 | |
| CodeV-R1-7BType=Verilog-Specific Reasoning Models2025.11 | 69.9 | 79.3 | 81.9 | |
| CraftRTLBase Model=Starcoder2 15B, Training Strategy=Training-Based2026.03 | 68 | 72.4 | 74.6 | |
| CraftRTLBackbone=Starcoder2, Size=15B2025.07 | 68 | 72.4 | 74.6 | |
| CodeV-R1-Distill-7BType=Verilog-Specific Reasoning Models2025.11 | 65.7 | 76.8 | 79.7 | |
| CraftRTLBase Model=DeepSeek-Coder 6.7B, Training Strategy=Training-Based2026.03 | 65.4 | 70 | 72.1 | |
| CraftRTLBackbone=DeepSeek-Coder, Size=6.7B2025.07 | 65.4 | 70 | 72.1 | |
| QiMeng-CRUX-Final-7BType=Verilog-Specific General Models2025.11 | 65.2 | 72 | 73.8 | |
| ChipSeekBackbone=Deepseek-Coder, Size=7B2025.07 | 64.3 | 71.1 | 73.7 | |
| ChipSeekBackbone=CodeQwen, Size=7B2025.07 | 63.8 | 69.4 | 70.5 | |
| QWQ-32BType=Foundation Reasoning Models2025.11 | 63.6 | 78 | 81.3 | |
| ChipSeekBackbone=CodeLlama, Size=7B2025.07 | 63.4 | 70.1 | 72.4 | |
| QiMeng-CRUX-SFT-7BType=Verilog-Specific General Models2025.11 | 63.2 | 73.1 | 76.1 | |
| CraftRTLBase Model=CodeLlama 7B, Training Strategy=Training-Based2026.03 | 63.1 | 67.8 | 69.7 | |
| CraftRTLBackbone=CodeLlama, Size=7B2025.07 | 63.1 | 67.8 | 69.7 | |
| ChipSeek-R1Base Model=Qwen2.5-Coder 7B, Training Strategy=Training-Based2026.03 | 62.2 | 73.7 | 76.9 | |
| ChipSeekBackbone=Qwen2.5-Coder, Size=7B2025.07 | 62.2 | 73.7 | 76.9 | |
| HaVen-7BType=Verilog-Specific Reasoning Models2025.11 | 61.1 | 64.8 | — | |
| GPT-4oType=Foundation General Models2025.11 | 60.1 | 71.4 | 74.5 | |
| GPT-o1Type=Foundational Models, Size=-2025.07 | 58.5 | 68 | 71.2 | |
| CodeV-Qwen2.5-7BType=Verilog-Specific General Models2025.11 | 57.9 | 66.7 | 69.7 | |
| ReasoningVBackbone=Qwen2.5-Coder, Size=7B2025.07 | 57.8 | 69.3 | 72.4 | |
| DirectBase Model=GPT-4o, Training Strategy=Training-Free2026.03 | 57.1 | 63.9 | 66.7 | |
| GPT-4oType=Foundational Models, Size=-2025.07 | 57.1 | 63.9 | 66.7 | |
| OriGen-7BType=Verilog-Specific General Models2025.11 | 54.4 | 60.1 | 64.2 | |
| OriGenBase Model=DeepSeek-Coder 6.7B, Training Strategy=Training-Based2026.03 | 54.4 | 60.1 | 64.2 | |
| OrigenBackbone=DeepSeek-Coder, Size=7B2025.07 | 54.4 | 60.1 | 64.2 | |
| CodeVBase Model=CodeQwen 7B, Training Strategy=Training-Based2026.03 | 53.2 | 65.1 | 68.5 | |
| CodeVBackbone=CodeQwen, Size=7B2025.07 | 53.2 | 65.1 | 68.5 | |
| CodeV-Qwen1.5-7BType=Verilog-Specific General Models2025.11 | 52.7 | 62.5 | 67.3 | |
| CodeVBase Model=DeepSeek-Coder 6.7B, Training Strategy=Training-Based2026.03 | 52.7 | 62.5 | 67.3 | |
| CodeVBackbone=DeepSeek-Coder, Size=6.7B2025.07 | 52.7 | 62.5 | 67.3 | |
| VeriPrefer-7BType=Verilog-Specific General Models2025.11 | 49.7 | 62.3 | — | |
| Qwen2.5-Coder-32BType=General Code Models2025.11 | 47.6 | 58.1 | 61.8 | |
| BetterVBase Model=CodeQwen 7B, Training Strategy=Training-Based2026.03 | 46.1 | 53.7 | 58.2 | |
| BetterVBase Model=DeepSeek-Coder 6.7B, Training Strategy=Training-Based2026.03 | 45.9 | 53.3 | 57.6 | |
| CodeVBase Model=CodeLlama 7B, Training Strategy=Training-Based2026.03 | 45.2 | 59.5 | 63.8 | |
| CodeVBackbone=CodeLlama, Size=7B2025.07 | 45.2 | 59.5 | 63.8 | |
| RTLCoder-6.7BType=Verilog-Specific General Models2025.11 | 41.6 | 50.1 | 53.4 | |
| RTLCoderBase Model=DeepSeek-Coder 7B, Training Strategy=Training-Based2026.03 | 41.6 | 50.1 | 53.4 | |
| RTLCoderBackbone=DeepSeek-Coder, Size=7B2025.07 | 41.6 | 50.1 | 53.4 | |
| BetterVBase Model=CodeLlama 7B, Training Strategy=Training-Based2026.03 | 40.9 | 50 | 53.3 | |
| RTLCoderBase Model=Mistral 7B, Training Strategy=Training-Based2026.03 | 36.7 | 45.5 | 49.2 | |
| RTLCoderBackbone=Mistral, Size=7B2025.07 | 36.7 | 45.5 | 49.2 | |
| Deepseek-Coder-6.7BType=General Code Models2025.11 | 30.2 | 33.9 | 34.9 | |
| DeepSeek-CoderType=Base Models, Size=6.7B2025.07 | 30.2 | 33.9 | 34.9 | |
| Qwen2.5-CoderType=Base Models, Size=7B2025.07 | 27.8 | 43.6 | 48.7 | |
| Qwen2.5-Coder-7BType=General Code Models2025.11 | 22.9 | 36 | 39.5 | |
| CodeQwenType=Base Models, Size=7B2025.07 | 22.5 | 26.1 | 28 | |
| CodeLlamaType=Base Models, Size=7B2025.07 | 18.2 | 22.7 | 24.3 |