Code Summarization on VHDL
60.9LLM PRROSUM-MCTS
Evaluation results
| Method | Links | ||
|---|---|---|---|
| ROSUM-MCTSLLM=GPT-4o2026.06 | 60.9 | 50.8 | |
| ROSUM-MCTSLLM=Llama-32026.06 | 53.2 | 45.9 | |
| CODESLLM=GPT-4o2026.06 | 52.3 | 43.2 | |
| VanillaLLM=GPT-4o2026.06 | 48.7 | 39.3 | |
| CODESLLM=Llama-32026.06 | 46.1 | 39.5 | |
| ROSUM-MCTSLLM=Granite Code-34b2026.06 | 45.6 | 39.3 | |
| VanillaLLM=Llama-32026.06 | 42.6 | 34.1 | |
| CODESLLM=Granite Code-34b2026.06 | 39.1 | 33.4 | |
| ROSUM-MCTSLLM=DeepSeek-Coder-33b2026.06 | 35.7 | 33.1 | |
| VanillaLLM=Granite Code-34b2026.06 | 34.2 | 29.5 | |
| CODESLLM=DeepSeek-Coder-33b2026.06 | 29.4 | 28.6 | |
| VanillaLLM=DeepSeek-Coder-33b2026.06 | 26.8 | 25.1 |