Specification-to-RTL on VerilogEval SR v2 (test)
77.5Pass@1DeepSeek-R1-671B
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| DeepSeek-R1-671BOpen source=true2025.05 | 77.5 | 84.7 | 87.4 | |
| CodeV-R1-7BOpen source=true2025.05 | 68.8 | 78.2 | 81.1 | |
| CodeV-R1-7B-DistillOpen source=true2025.05 | 65.2 | 75.2 | 77.5 | |
| QWQ-32BOpen source=true2025.05 | 64.2 | 77.3 | 80.1 | |
| GPT-4oOpen source=false2025.05 | 64.1 | 73.7 | 76.2 | |
| DeepSeek-V3-671BOpen source=true2025.05 | 62.4 | 71.7 | 75 | |
| Qwen2.5-Coder-32B-InstructOpen source=true2025.05 | 47.5 | 60.7 | 64.7 | |
| DeepSeek-R1-Distill-Qwen-32BOpen source=true2025.05 | 43.9 | 63.3 | 69.2 | |
| Qwen2.5-Coder-7B-InstructOpen source=true2025.05 | 31.3 | 49.3 | 54.6 | |
| RTLCoder-DS-6.7BOpen source=true2025.05 | 31.1 | 47.8 | 52.3 | |
| DeepSeek-R1-Distill-Qwen-7BOpen source=true2025.05 | 0.6 | 2.2 | 3.5 |