RTL Synthesis on VHDL-Eval
84.2Pass@1SYMDIREC-GT (oracle)
Evaluation Results
| Method | Links | |
|---|---|---|
| SYMDIREC-GT (oracle)LLM=GPT-4o2026.03 | 84.2 | |
| SYMDIREC-GT (oracle)LLM=Llama-32026.03 | 72.1 | |
| SYMDIRECLLM=GPT-4o2026.03 | 63.4 | |
| SYMDIRECLLM=Llama-32026.03 | 54.5 | |
| VRAG-FTLLM=GPT-4o2026.03 | 53.1 | |
| VRAG-CodeBERTLLM=GPT-4o2026.03 | 48.7 | |
| VRAG-FTLLM=Llama-32026.03 | 43.9 | |
| VRAG-CodeBERTLLM=Llama-32026.03 | 39.6 | |
| ReAct PromptingLLM=GPT-4o2026.03 | 35.3 | |
| CoDesLLM=GPT-4o2026.03 | 34.8 | |
| ReAct PromptingLLM=Llama-32026.03 | 29.1 | |
| Vanilla PromptingLLM=GPT-4o2026.03 | 28.5 | |
| CoDesLLM=Llama-32026.03 | 27.4 | |
| Vanilla PromptingLLM=Llama-32026.03 | 22.6 |