Code Summarization Factual Consistency on Python
0.497Pearson Correlation (rp)ReFEree
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| ReFEreeCategory=reference-free, Context Info=w/ info2026.04 | 0.497 | 0.489 | 0.39 | 0.459 | |
| ReFEreeCategory=reference-free, Context Info=w/o info2026.04 | 0.432 | 0.432 | 0.349 | 0.404 | |
| G-EvalCategory=reference-free2026.04 | 0.427 | 0.413 | 0.36 | 0.4 | |
| LLM-judgeCategory=reference-free2026.04 | 0.419 | 0.41 | 0.36 | 0.396 | |
| CODERPECategory=reference-free2026.04 | 0.418 | 0.405 | 0.353 | 0.392 | |
| FactscoreCategory=reference-free2026.04 | 0.41 | 0.426 | 0.338 | 0.391 | |
| SBCSCategory=reference-based2026.04 | 0.068 | 0.036 | 0.028 | 0.044 | |
| ROUGE-1Category=reference-based2026.04 | 0.046 | 0.044 | 0.034 | 0.041 | |
| ROUGE-LCategory=reference-based2026.04 | 0.045 | 0.037 | 0.029 | 0.037 | |
| METEORCategory=reference-based2026.04 | 0.033 | 0.024 | 0.019 | 0.025 | |
| ROUGE-2Category=reference-based2026.04 | 0.03 | 0.031 | 0.024 | 0.028 | |
| BLEUCategory=reference-based2026.04 | 0.016 | 0.018 | 0.015 | 0.017 | |
| BERTScoreCategory=reference-based2026.04 | 0.015 | 0 | 0 | 0.005 | |
| SBEDCategory=reference-based2026.04 | -0.058 | -0.036 | -0.028 | -0.041 | |
| SIDECategory=reference-free2026.04 | -0.064 | -0.056 | -0.043 | -0.054 |