RTL generation on VerilogEval 156 cases (test)
0.78Pass@1Claude-4.5-Sonnet
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Claude-4.5-Sonnettype=Commercial Model2025.11 | 0.78 | 0.84 | 0.86 | |
| GPT-5type=Commercial Model2025.11 | 0.7 | 0.76 | 0.79 | |
| VeriDispatcher (Average+OpenAI Emb.-w)Embedding Type=Average+OpenAI, Commercial Model Inclusion=Included, Classifier=MLP2025.11 | 0.69 | 0.76 | 0.8 | |
| VeriDispatcher (OpenAI Emb.-w)Embedding Type=OpenAI, Commercial Model Inclusion=Included, Classifier=MLP2025.11 | 0.69 | 0.78 | 0.79 | |
| VeriDispatcher (Qwen Emb.-w)Embedding Type=Qwen, Commercial Model Inclusion=Included, Classifier=MLP2025.11 | 0.65 | 0.74 | 0.79 | |
| CodeV-R1-Distill-Qwen-7B2025.11 | 0.52 | 0.65 | 0.7 | |
| HaVen2025.11 | 0.48 | 0.56 | 0.6 | |
| CodeV-DS-6.7B2025.11 | 0.46 | 0.64 | 0.71 | |
| CodeV-QW-7B2025.11 | 0.45 | 0.65 | 0.69 | |
| VeriDispatcher (Average Emb.-wo)Embedding Type=Average, Commercial Model Inclusion=Excluded, Classifier=MLP2025.11 | 0.43 | 0.62 | 0.69 | |
| VeriDispatcher (Qwen Emb.-wo)Embedding Type=Qwen, Commercial Model Inclusion=Excluded, Classifier=MLP2025.11 | 0.43 | 0.61 | 0.65 | |
| VeriDispatcher (OpenAI Emb.-wo)Embedding Type=OpenAI, Commercial Model Inclusion=Excluded, Classifier=MLP2025.11 | 0.38 | 0.6 | 0.65 | |
| DeepSeek-7BForget sets=20%2025.12 | 0.38 | 0.44 | — | |
| DeepSeek-7BForget sets=10%2025.12 | 0.37 | 0.45 | — | |
| RTLCoder-Mistral2025.11 | 0.36 | 0.5 | 0.56 | |
| Qwen-7BForget sets=0%2025.12 | 0.36 | 0.49 | — | |
| Qwen-7BForget sets=10%2025.12 | 0.36 | 0.48 | — | |
| Llama-8BForget sets=0%2025.12 | 0.35 | 0.46 | — | |
| DeepSeek-7BForget sets=0%2025.12 | 0.35 | 0.44 | — | |
| Qwen-7BForget sets=20%2025.12 | 0.34 | 0.48 | — | |
| OriGen2025.11 | 0.33 | 0.56 | 0.63 | |
| DeepSeek-7BForget sets=30%2025.12 | 0.33 | 0.42 | — | |
| Qwen-7BForget sets=30%2025.12 | 0.33 | 0.44 | — | |
| CodeV-CL-7B2025.11 | 0.31 | 0.56 | 0.62 | |
| Llama-8BForget sets=10%2025.12 | 0.31 | 0.45 | — | |
| Llama-8BForget sets=20%2025.12 | 0.31 | 0.43 | — | |
| Llama-7BForget sets=0%2025.12 | 0.3 | 0.39 | — | |
| Llama-7BForget sets=10%2025.12 | 0.3 | 0.38 | — | |
| Llama-7BForget sets=20%2025.12 | 0.29 | 0.39 | — | |
| Llama-7BForget sets=30%2025.12 | 0.28 | 0.37 | — | |
| Llama-8BForget sets=30%2025.12 | 0.28 | 0.38 | — | |
| RTLCoder-DeepSeek2025.11 | 0.26 | 0.51 | 0.58 |