Hardware Generation on VerilogEval 156 problems
99.4Compilation RateCKTFORMALIZER (synth)
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| CKTFORMALIZER (synth)loop type=iterative compile-fix with Lean type checking, synthesis feedback=Yosys synthesis in agent loop2026.05 | 99.4 | 69.9 | 70.3 | 99.4 | 95.5 | 95.5 | 95.5 | 68.6 | 66.7 | 96.5 | |
| CKTFORMALIZERloop type=iterative compile-fix with Lean type checking2026.05 | 92.3 | 72.4 | 78.5 | 91 | 91 | 91 | 91 | 71.2 | 71.8 | 91 | |
| CodeVmode=instruction-tuned2026.05 | 91.7 | 42.3 | 46.2 | 41.7 | 40.4 | 40.4 | 40.4 | 41.7 | 40.4 | 40.7 | |
| Direct SVmode=single LLM call2026.05 | 89.7 | 71.8 | 80 | 71.8 | 67.9 | 67.9 | 67.9 | 67.9 | 65.4 | 68.9 | |
| RTLCodermodel size=7B, mode=fine-tuned2026.05 | 82.1 | 43.6 | 53.1 | 43.6 | 42.3 | 42.3 | 42.3 | 43.6 | 42.3 | 42.6 |