Function-level code summarization on Selected repositories Obfuscated code (test)
16.2ROUGE-LAgent4cs
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Agent4csModel=Gemini-2.5-flash2026.07 | 16.2 | 15.6 | 79.2 | 62.2 | 74.2 | 3.35 | 3.9 | 3.23 | 3.91 | |
| Agent4csModel=GPT-4o2026.07 | 16.1 | 16.8 | 81.9 | 53 | 76.2 | 3.35 | 3.9 | 3.08 | 3.88 | |
| Baseline (CS-BF)Model=GPT-52026.07 | 16 | 17.9 | 81.3 | 60.7 | 69.9 | 3.86 | 3.99 | 3.81 | 3.99 | |
| CS-BFModel=Gemini-2.5-flash2026.07 | 16 | 15.3 | 78.8 | 61.8 | 73.6 | 3.38 | 3.92 | 3.21 | 3.92 | |
| Agent4csModel=GPT-4.12026.07 | 15.7 | 17.3 | 81.7 | 67.1 | 78.9 | 3.53 | 3.96 | 3.2 | 3.93 | |
| Baseline (HR-CS)Model=GPT-52026.07 | 15.6 | 18.1 | 80.9 | 59.9 | 69.8 | 3.86 | 3.99 | 3.84 | 3.99 | |
| CS-BFModel=GPT-4o2026.07 | 15.6 | 17.3 | 81.6 | 53.5 | 77 | 3.37 | 3.88 | 3.1 | 3.86 | |
| Agent4csModel=GPT-52026.07 | 15.4 | 18.2 | 81.9 | 61.2 | 70.4 | 3.88 | 3.99 | 3.84 | 4 | |
| CS-BFModel=GPT-4.12026.07 | 15.2 | 17.9 | 81.2 | 64.5 | 77.5 | 3.55 | 3.94 | 3.22 | 3.91 | |
| HR-CSModel=GPT-4o2026.07 | 15 | 15.9 | 81.4 | 50.9 | 76.3 | 3.34 | 3.88 | 3.02 | 3.86 | |
| HR-CSModel=GPT-4.12026.07 | 14.9 | 16.3 | 81.5 | 62.3 | 72.3 | 3.31 | 3.96 | 3.1 | 3.92 | |
| HR-CSModel=Gemini-2.5-flash2026.07 | 14.9 | 15.9 | 79.8 | 61.4 | 72.8 | 3.25 | 3.87 | 3.01 | 3.85 | |
| CS-BFModel=LLaMA-3.1-8B2026.07 | 14.9 | 13.6 | 81.3 | 60.9 | 70.3 | 3.01 | 3.46 | 2.76 | 3.49 | |
| Agent4csModel=LLaMA-3.1-8B2026.07 | 14.4 | 13.9 | 80.8 | 61.3 | 70.8 | 2.98 | 3.48 | 2.78 | 3.46 | |
| HR-CSModel=LLaMA-3.1-8B2026.07 | 14 | 13.3 | 79.9 | 60.3 | 67.9 | 3.12 | 3.58 | 2.76 | 3.49 | |
| Agent4csModel=Gemma3-4B2026.07 | 13 | 13.2 | 79 | 57.8 | 70.7 | 3.24 | 3.63 | 2.84 | 3.58 | |
| HR-CSModel=Gemma3-4B2026.07 | 12.9 | 12.9 | 78.7 | 57 | 68.8 | 3.18 | 3.54 | 2.68 | 3.49 | |
| CS-BFModel=Gemma3-4B2026.07 | 12.7 | 13.7 | 79.5 | 58 | 71.2 | 3.26 | 3.66 | 2.86 | 3.56 | |
| CS-BFModel=Qwen3-8B2026.07 | 10.8 | 10.7 | 78.9 | 56.2 | 72.7 | 3.41 | 3.91 | 3.14 | 3.89 | |
| HR-CSModel=Qwen3-8B2026.07 | 10.5 | 10.8 | 78.6 | 57.1 | 72.3 | 3.24 | 3.82 | 3.07 | 3.86 | |
| Agent4csModel=Qwen3-8B2026.07 | 10.3 | 10.2 | 79.4 | 56.6 | 73.2 | 3.39 | 3.89 | 3.12 | 3.87 |