Hierarchical Code Summarization on Hierarchical Folder Structure
79.4Semantic SimilarityAgent4cs
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Agent4csBackbone=GPT-5, Depth=Average2026.07 | 79.4 | 80.9 | |
| Agent4csBackbone=GPT-4o, Depth=Layer 102026.07 | 77.6 | 87.3 | |
| HR-CSBackbone=GPT-5, Depth=Average2026.07 | 73.8 | 74.8 | |
| HR-CSBackbone=LLaMA-3.1-8B, Depth=Average2026.07 | 72.7 | 82.7 | |
| CS-BFBackbone=GPT-5, Depth=Average2026.07 | 72.1 | 73.2 | |
| CS-BFBackbone=LLaMA-3.1-8B, Depth=Average2026.07 | 71.4 | 84.6 | |
| HR-CSBackbone=GPT-4o, Depth=Layer 102026.07 | 71 | 72.5 | |
| Agent4csBackbone=LLaMA-3.1-8B, Depth=Average2026.07 | 68.3 | 84.3 | |
| Agent4csBackbone=GPT-4o, Depth=Average2026.07 | 68.1 | 82.9 | |
| CS-BFBackbone=GPT-4.1, Depth=Average2026.07 | 66.7 | 68.6 | |
| CS-BFBackbone=GPT-4o, Depth=Average2026.07 | 65.1 | 68.2 | |
| Agent4csBackbone=Qwen3-8B, Depth=Average2026.07 | 64.4 | 87 | |
| HR-CSBackbone=GPT-4o, Depth=Average2026.07 | 63.2 | 65.9 | |
| CS-BFBackbone=Gemini-2.5-flash, Depth=Average2026.07 | 58.5 | 58.2 | |
| HR-CSBackbone=GPT-4o, Depth=Layer 12026.07 | 58.3 | 59.4 | |
| Agent4csBackbone=GPT-4o, Depth=Layer 12026.07 | 57.6 | 75.1 | |
| HR-CSBackbone=Qwen3-8B, Depth=Average2026.07 | 56.4 | 87.2 | |
| CS-BFBackbone=Qwen3-8B, Depth=Average2026.07 | 54.9 | 84.1 | |
| Agent4csBackbone=Gemma3-4B, Depth=Average2026.07 | 46 | 64.3 | |
| CS-BFBackbone=Gemma3-4B, Depth=Average2026.07 | 41.2 | 54.3 | |
| HR-CSBackbone=Gemma3-4B, Depth=Average2026.07 | 40.2 | 52.6 |