RTL Design Code Generation on VerilogHuman v2
98.1Syntax AccuracyClaude-3.7
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Claude-3.7Evaluation Protocol=Original pass@12026.05 | 98.1 | 76.3 | |
| GPT-4oEvaluation Protocol=RTL-BenchMT fixes2026.05 | 98.1 | 71.4 | |
| Claude-3.7Evaluation Protocol=RTL-BenchMT fixes2026.05 | 98.1 | 76.9 | |
| GPT-4o-miniEvaluation Protocol=Original pass@12026.05 | 96.2 | 51.9 | |
| GPT-4oEvaluation Protocol=Original pass@12026.05 | 96.2 | 68.8 | |
| LlaMA-405BEvaluation Protocol=Original pass@12026.05 | 91.7 | 60.9 | |
| LlaMA-70BEvaluation Protocol=Original pass@12026.05 | 80.1 | 42.9 | |
| QwQ-32BEvaluation Protocol=Original pass@12026.05 | 61.5 | 55.1 | |
| GPT-3.5Evaluation Protocol=Original pass@12026.05 | 53.2 | 26.3 |