RTL Synthesis on VHDL-Eval (Pass@1, Reasoning Fidelity (%))
78.6Pass@1StepPRM-RTL (Full)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| StepPRM-RTL (Full)Training/Evaluation Protocol=Our Model Variants2026.06 | 78.6 | 80.2 | |
| No MCTS (Sampling-Only)Training/Evaluation Protocol=Our Model Variants2026.06 | 73.8 | 76.5 | |
| Supervised RAFT OnlyTraining/Evaluation Protocol=Our Model Variants2026.06 | 72.1 | 73 | |
| No PRMTraining/Evaluation Protocol=Our Model Variants2026.06 | 70.9 | 70.8 | |
| RAG-FT (GPT-4o)Training/Evaluation Protocol=Finetuning-based Models2026.06 | 53.1 | — | |
| RAG-CodeBERT (GPT-4o)Training/Evaluation Protocol=Finetuning-based Models2026.06 | 48.7 | — | |
| CoDes (GPT-4o)Training/Evaluation Protocol=Prompt-based Models2026.06 | 34.8 | — | |
| Vanilla Prompting (GPT-4o)Training/Evaluation Protocol=Prompt-based Models2026.06 | 28.5 | — |