Code Summarization Factual Consistency on Java
0.515Pearson (rp)ReFEree
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| ReFEreeCategory=reference-free, Context Info=w/ info2026.04 | 0.515 | 0.502 | 0.423 | 0.48 | |
| ReFEreeCategory=reference-free, Context Info=w/o info2026.04 | 0.469 | 0.458 | 0.387 | 0.438 | |
| G-EvalCategory=reference-free2026.04 | 0.457 | 0.407 | 0.355 | 0.406 | |
| CODERPECategory=reference-free2026.04 | 0.457 | 0.398 | 0.347 | 0.401 | |
| FactscoreCategory=reference-free2026.04 | 0.391 | 0.365 | 0.31 | 0.355 | |
| LLM-judgeCategory=reference-free2026.04 | 0.385 | 0.363 | 0.318 | 0.355 | |
| SBCSCategory=reference-based2026.04 | 0.363 | 0.227 | 0.178 | 0.256 | |
| ROUGE-LCategory=reference-based2026.04 | 0.202 | 0.178 | 0.137 | 0.172 | |
| ROUGE-1Category=reference-based2026.04 | 0.201 | 0.186 | 0.14 | 0.176 | |
| ROUGE-2Category=reference-based2026.04 | 0.194 | 0.176 | 0.137 | 0.169 | |
| BERTScoreCategory=reference-based2026.04 | 0.189 | 0.149 | 0.113 | 0.15 | |
| METEORCategory=reference-based2026.04 | 0.135 | 0.139 | 0.105 | 0.126 | |
| BLEUCategory=reference-based2026.04 | 0.081 | 0.065 | 0.058 | 0.068 | |
| SIDECategory=reference-free2026.04 | 0.032 | 0.024 | 0.019 | 0.025 | |
| SBEDCategory=reference-based2026.04 | -0.348 | -0.227 | -0.178 | -0.251 |