RTL Summarization on VHDL-Eval
64.7ROUGE-LSYMDIREC-GT (oracle)
Evaluation results
| Method | Links | |
|---|---|---|
| SYMDIREC-GT (oracle)LLM=GPT-4o2026.03 | 64.7 | |
| SYMDIREC-GT (oracle)LLM=Llama-32026.03 | 57.9 | |
| SYMDIRECLLM=GPT-4o2026.03 | 56.6 | |
| VRAG-FTLLM=GPT-4o2026.03 | 52.8 | |
| SYMDIRECLLM=Llama-32026.03 | 50.8 | |
| VRAG-CodeBERTLLM=GPT-4o2026.03 | 50.3 | |
| VRAG-FTLLM=Llama-32026.03 | 48.1 | |
| VRAG-CodeBERTLLM=Llama-32026.03 | 44.5 | |
| CoDesLLM=GPT-4o2026.03 | 43.2 | |
| ReAct PromptingLLM=GPT-4o2026.03 | 43 | |
| CoDesLLM=Llama-32026.03 | 39.5 | |
| Vanilla PromptingLLM=GPT-4o2026.03 | 39.3 | |
| ReAct PromptingLLM=Llama-32026.03 | 38.8 | |
| Vanilla PromptingLLM=Llama-32026.03 | 34.1 |