RTL Design Code Generation on VerilogMachine v1
96.5Syntax AccuracyGPT-4o
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GPT-4oEvaluation Protocol=Original pass@12026.05 | 96.5 | 74.1 | |
| GPT-4oEvaluation Protocol=RTL-BenchMT fixes2026.05 | 96.5 | 77.6 | |
| Claude-3.7Evaluation Protocol=Original pass@12026.05 | 92.3 | 76.9 | |
| Claude-3.7Evaluation Protocol=RTL-BenchMT fixes2026.05 | 92.3 | 78.2 | |
| GPT-4o-miniEvaluation Protocol=Original pass@12026.05 | 87.8 | 59.6 | |
| LlaMA-405BEvaluation Protocol=Original pass@12026.05 | 81.4 | 62.2 | |
| LlaMA-70BEvaluation Protocol=Original pass@12026.05 | 78.8 | 47.4 | |
| QwQ-32BEvaluation Protocol=Original pass@12026.05 | 74.4 | 61.5 | |
| GPT-3.5Evaluation Protocol=Original pass@12026.05 | 25 | 16 |