RTL Synthesis on Verilog-Eval
90.2Pass@1SYMDIREC-GT (oracle)
Evaluation results
| Method | Links | |
|---|---|---|
| SYMDIREC-GT (oracle)LLM=GPT-4o2026.03 | 90.2 | |
| SYMDIREC-GT (oracle)LLM=Llama-32026.03 | 80.7 | |
| SYMDIRECLLM=GPT-4o2026.03 | 80.5 | |
| VRAG-FTLLM=GPT-4o2026.03 | 71.9 | |
| VRAG-CodeBERTLLM=GPT-4o2026.03 | 68.8 | |
| SYMDIRECLLM=Llama-32026.03 | 65.2 | |
| RTLCoder (open-source)LLM=Mistral2026.03 | 62.5 | |
| ReAct PromptingLLM=GPT-4o2026.03 | 61.6 | |
| CoDesLLM=GPT-4o2026.03 | 60.2 | |
| VRAG-FTLLM=Llama-32026.03 | 56.9 | |
| Vanilla PromptingLLM=GPT-4o2026.03 | 54.3 | |
| CodeV (instruction-tuned)LLM=CodeQwen2026.03 | 53.2 | |
| VRAG-CodeBERTLLM=Llama-32026.03 | 52.7 | |
| ReAct PromptingLLM=Llama-32026.03 | 43.7 | |
| CoDesLLM=Llama-32026.03 | 43.5 | |
| Vanilla PromptingLLM=Llama-32026.03 | 38.5 |