HDL generation on VerilogEval 2.0
84.97Pass@1VERIMOA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| VERIMOAModel=GPT-4o2025.10 | 84.97 | 89.65 | 91.03 | |
| VERIMOAModel=Qwen2.5-Coder-32B2025.10 | 73.31 | 79.05 | 81.27 | |
| VERIMOAModel=GPT-4o-mini2025.10 | 72.43 | 76.94 | 79.46 | |
| VERIMOAModel=Qwen2.5-32B2025.10 | 71.85 | 76.48 | 78.16 | |
| VeriMaASModel=GPT-4o2025.10 | 71.34 | 76.12 | 79.21 | |
| HDLCoReModel=GPT-4o2025.10 | 69.6 | 73.53 | 75.86 | |
| VERIMOAModel=Qwen2.5-Coder-14B2025.10 | 66.86 | 73.24 | 76.38 | |
| CoTModel=GPT-4o2025.10 | 66.38 | 71.18 | 74.22 | |
| VERIMOAModel=Qwen2.5-14B2025.10 | 65.27 | 71.61 | 74.64 | |
| DirectModel=GPT-4o2025.10 | 64.74 | 69.89 | 71.66 | |
| VERIMOAModel=Qwen2.5-Coder-7B2025.10 | 60.96 | 68.13 | 70.69 | |
| VeriMaASModel=GPT-4o-mini2025.10 | 57.24 | 64.77 | 66.85 | |
| VeriMaASModel=Qwen2.5-Coder-32B2025.10 | 56.67 | 63.46 | 66.92 | |
| VERIMOAModel=Qwen2.5-7B2025.10 | 56.44 | 62.07 | 65.27 | |
| VeriMaASModel=Qwen2.5-32B2025.10 | 53.57 | 61.82 | 64.26 | |
| HDLCoReModel=GPT-4o-mini2025.10 | 53.5 | 61.46 | 63.72 | |
| CoTModel=GPT-4o-mini2025.10 | 52.2 | 59.83 | 60.93 | |
| HDLCoReModel=Qwen2.5-32B2025.10 | 51.75 | 59.63 | 61.79 | |
| HDLCoReModel=Qwen2.5-Coder-32B2025.10 | 51.28 | 58.63 | 61.47 | |
| VeriMaASModel=Qwen2.5-Coder-14B2025.10 | 50.96 | 58.48 | 62.63 | |
| VeriMaASModel=Qwen2.5-14B2025.10 | 48.97 | 57.84 | 61.77 | |
| DirectModel=GPT-4o-mini2025.10 | 48.97 | 56.94 | 58.62 | |
| CoTModel=Qwen2.5-32B2025.10 | 48.72 | 57.25 | 59.8 | |
| CoTModel=Qwen2.5-Coder-32B2025.10 | 48.65 | 56.31 | 59.34 | |
| DirectModel=Qwen2.5-Coder-32B2025.10 | 46.93 | 55.73 | 57.12 | |
| DirectModel=Qwen2.5-32B2025.10 | 46.85 | 56.11 | 58.41 | |
| HDLCoReModel=Qwen2.5-Coder-14B2025.10 | 46.82 | 54.79 | 57.04 | |
| HDLCoReModel=Qwen2.5-14B2025.10 | 45.18 | 55.83 | 59.42 | |
| VeriMaASModel=Qwen2.5-Coder-7B2025.10 | 44.73 | 51.62 | 55.78 | |
| CoTModel=Qwen2.5-Coder-14B2025.10 | 43.81 | 51.27 | 54.11 | |
| CoTModel=Qwen2.5-14B2025.10 | 41.24 | 51.62 | 54.37 | |
| HDLCoReModel=Qwen2.5-Coder-7B2025.10 | 40.67 | 47.81 | 52.64 | |
| DirectModel=Qwen2.5-Coder-14B2025.10 | 39.74 | 49.72 | 52.21 | |
| DirectModel=Qwen2.5-14B2025.10 | 39.26 | 49.98 | 53.74 | |
| CoTModel=Qwen2.5-Coder-7B2025.10 | 36.74 | 45.88 | 50.99 | |
| DirectModel=Qwen2.5-Coder-7B2025.10 | 33.91 | 43.9 | 48.06 | |
| VeriMaASModel=Qwen2.5-7B2025.10 | 32.81 | 40.25 | 49.07 | |
| HDLCoReModel=Qwen2.5-7B2025.10 | 30.92 | 37.86 | 46.49 | |
| CoTModel=Qwen2.5-7B2025.10 | 26.87 | 35.33 | 42.58 | |
| DirectModel=Qwen2.5-7B2025.10 | 22.9 | 32.2 | 40.39 |