RTL Summarization on Verilog-Eval
70.2ROUGE-LSYMDIREC-GT (oracle)
Evaluation results
| Method | Links | |
|---|---|---|
| SYMDIREC-GT (oracle)LLM=GPT-4o2026.03 | 70.2 | |
| SYMDIREC-GT (oracle)LLM=Llama-32026.03 | 63.3 | |
| SYMDIRECLLM=GPT-4o2026.03 | 62.5 | |
| VRAG-FTLLM=GPT-4o2026.03 | 57 | |
| SYMDIRECLLM=Llama-32026.03 | 56.1 | |
| VRAG-CodeBERTLLM=GPT-4o2026.03 | 53.2 | |
| VRAG-FTLLM=Llama-32026.03 | 50.5 | |
| VRAG-CodeBERTLLM=Llama-32026.03 | 47.4 | |
| CoDesLLM=GPT-4o2026.03 | 46.9 | |
| ReAct PromptingLLM=GPT-4o2026.03 | 46.1 | |
| CoDesLLM=Llama-32026.03 | 43.5 | |
| Vanilla PromptingLLM=GPT-4o2026.03 | 43.1 | |
| ReAct PromptingLLM=Llama-32026.03 | 42.9 | |
| Vanilla PromptingLLM=Llama-32026.03 | 40.2 |