Verilog Code Generation on VerilogEval SR v2
83.3Pass@1Deepseek-R1-671B
Evaluation Results
| Method | Links | |
|---|---|---|
| Deepseek-R1-671BSetting=T=02025.11 | 83.3 | |
| Deepseek-R1-671BSetting=T=0.82025.11 | 79.8 | |
| Deepseek-R1-671BSetting=T-cross2025.11 | 77.5 | |
| Deepseek-V3-671BSetting=T=02025.11 | 68.8 | |
| CodeV-R1-7BSetting=T-cross2025.11 | 68.8 | |
| Deepseek-V3-671BSetting=T=0.82025.11 | 66.9 | |
| CodeV-R1-Distill-7BSetting=T-cross2025.11 | 65.2 | |
| QiMeng-CRUX-Final-7BSetting=T=02025.11 | 64.7 | |
| QiMeng-CRUX-Final-7BSetting=T=0.82025.11 | 64.4 | |
| QiMeng-CRUX-Final-7BSetting=T-cross2025.11 | 64.2 | |
| GPT-4oSetting=T-cross2025.11 | 64.1 | |
| GPT-4oSetting=T=02025.11 | 62.5 | |
| Deepseek-V3-671BSetting=T-cross2025.11 | 62.4 | |
| GPT-4oSetting=T=0.82025.11 | 61.4 | |
| QiMeng-CRUX-SFT-7BSetting=T-cross2025.11 | 61.1 | |
| QiMeng-CRUX-SFT-7BSetting=T=02025.11 | 59.6 | |
| QiMeng-CRUX-SFT-7BSetting=T=0.82025.11 | 57 | |
| HaVen-7BSetting=T-cross2025.11 | 54.6 | |
| OriGen-7BSetting=T=02025.11 | 49.3 | |
| Qwen2.5-Coder-32BSetting=T-cross2025.11 | 47.5 | |
| Qwen2.5-Coder-32BSetting=T=02025.11 | 46.8 | |
| OriGen-7BSetting=T=0.82025.11 | 46.8 | |
| CodeV-Qwen2.5-7BSetting=T=02025.11 | 44.8 | |
| CodeV-Qwen2.5-7BSetting=T-cross2025.11 | 42.2 | |
| Qwen2.5-Coder-32BSetting=T=0.82025.11 | 41.6 | |
| CodeV-Qwen2.5-7BSetting=T=0.82025.11 | 37.4 | |
| Deepseek-Coder-6.7BSetting=T=02025.11 | 36.8 | |
| RTLCoder-6.7BSetting=T=02025.11 | 36.8 | |
| Qwen2.5-Coder-7BSetting=T-cross2025.11 | 31.3 | |
| RTLCoder-6.7BSetting=T-cross2025.11 | 31.1 | |
| Deepseek-Coder-6.7BSetting=T=0.82025.11 | 30.9 | |
| RTLCoder-6.7BSetting=T=0.82025.11 | 30.9 | |
| Qwen2.5-Coder-7BSetting=T=02025.11 | 23 | |
| Qwen2.5-Coder-7BSetting=T=0.82025.11 | 22 |