RTL Design Code Generation on VerilogHuman v1
99.3Syntax AccuracyClaude-3.7
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Claude-3.7Evaluation Protocol=RTL-BenchMT fixes2026.05 | 99.3 | 79.8 | |
| GPT-4oEvaluation Protocol=RTL-BenchMT fixes2026.05 | 98.7 | 71.1 | |
| Claude-3.7Evaluation Protocol=Original pass@12026.05 | 98.1 | 80.1 | |
| GPT-4oEvaluation Protocol=Original pass@12026.05 | 96.8 | 67.3 | |
| GPT-4o-miniEvaluation Protocol=Original pass@12026.05 | 89.1 | 51.3 | |
| LlaMA-70BEvaluation Protocol=Original pass@12026.05 | 78.8 | 47.4 | |
| LlaMA-405BEvaluation Protocol=Original pass@12026.05 | 75 | 48.7 | |
| GPT-3.5Evaluation Protocol=Original pass@12026.05 | 73.7 | 32.7 | |
| QwQ-32BEvaluation Protocol=Original pass@12026.05 | 53.2 | 44.2 |