HDL Generation on RTLLM 2.0
69.17Pass@1VERIMOA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| VERIMOAModel=GPT-4o2025.10 | 69.17 | 73.68 | 75.62 | |
| VERIMOAModel=Qwen2.5-Coder-32B2025.10 | 65.49 | 71.42 | 74.11 | |
| VERIMOAModel=GPT-4o-mini2025.10 | 64.23 | 67.45 | 68.67 | |
| VERIMOAModel=Qwen2.5-32B2025.10 | 63.81 | 67.6 | 69.88 | |
| VeriMaASModel=GPT-4o2025.10 | 61.7 | 67.25 | 68.84 | |
| VERIMOAModel=Qwen2.5-Coder-14B2025.10 | 59.76 | 64.33 | 67.22 | |
| VeriMaASModel=GPT-4o-mini2025.10 | 57.25 | 61.46 | 63.17 | |
| HDLCoReModel=GPT-4o2025.10 | 56.27 | 61.47 | 63.23 | |
| VeriMaASModel=Qwen2.5-Coder-32B2025.10 | 55.82 | 59.97 | 62.31 | |
| VERIMOAModel=Qwen2.5-14B2025.10 | 55.06 | 62.76 | 66.33 | |
| CoTModel=GPT-4o2025.10 | 54.89 | 59.22 | 61.77 | |
| VERIMOAModel=Qwen2.5-Coder-7B2025.10 | 54.43 | 62.52 | 66.24 | |
| VeriMaASModel=Qwen2.5-32B2025.10 | 54.18 | 57.21 | 61.54 | |
| DirectModel=GPT-4o2025.10 | 52.48 | 56.18 | 57.62 | |
| VERIMOAModel=Qwen2.5-7B2025.10 | 52.07 | 58.1 | 60.46 | |
| HDLCoReModel=Qwen2.5-Coder-32B2025.10 | 51.72 | 54.73 | 59.64 | |
| VeriMaASModel=Qwen2.5-Coder-14B2025.10 | 51.2 | 58.93 | 61.67 | |
| HDLCoReModel=GPT-4o-mini2025.10 | 51.19 | 56.72 | 59.01 | |
| HDLCoReModel=Qwen2.5-32B2025.10 | 50.95 | 54.06 | 58.98 | |
| CoTModel=GPT-4o-mini2025.10 | 48.27 | 53.56 | 55.48 | |
| VeriMaASModel=Qwen2.5-14B2025.10 | 48.14 | 54.28 | 58.76 | |
| HDLCoReModel=Qwen2.5-Coder-14B2025.10 | 47.86 | 55.19 | 58.18 | |
| CoTModel=Qwen2.5-32B2025.10 | 46.99 | 50.5 | 52.46 | |
| DirectModel=GPT-4o-mini2025.10 | 46.6 | 50.71 | 51.95 | |
| CoTModel=Qwen2.5-Coder-32B2025.10 | 45.72 | 48.75 | 50.29 | |
| VeriMaASModel=Qwen2.5-Coder-7B2025.10 | 44.71 | 51.37 | 56.46 | |
| DirectModel=Qwen2.5-32B2025.10 | 43.62 | 48.73 | 50.75 | |
| HDLCoReModel=Qwen2.5-14B2025.10 | 41.78 | 49.97 | 53.17 | |
| CoTModel=Qwen2.5-Coder-14B2025.10 | 40.65 | 48.77 | 51.72 | |
| DirectModel=Qwen2.5-Coder-32B2025.10 | 40.4 | 45.42 | 48.24 | |
| VeriMaASModel=Qwen2.5-7B2025.10 | 40.33 | 47.24 | 50.47 | |
| CoTModel=Qwen2.5-14B2025.10 | 36.49 | 44.64 | 48.29 | |
| HDLCoReModel=Qwen2.5-Coder-7B2025.10 | 36.26 | 45.61 | 51.12 | |
| DirectModel=Qwen2.5-Coder-14B2025.10 | 35.61 | 42.34 | 46.43 | |
| HDLCoReModel=Qwen2.5-7B2025.10 | 34.68 | 40.32 | 44.82 | |
| CoTModel=Qwen2.5-Coder-7B2025.10 | 31.87 | 39.36 | 45.36 | |
| DirectModel=Qwen2.5-14B2025.10 | 31.71 | 40.2 | 45.87 | |
| CoTModel=Qwen2.5-7B2025.10 | 27.49 | 34.76 | 39.95 | |
| DirectModel=Qwen2.5-Coder-7B2025.10 | 26.16 | 36.94 | 41.11 | |
| DirectModel=Qwen2.5-7B2025.10 | 18.99 | 30.53 | 36.51 |