Summary Conciseness Evaluation on REALSumm
0.505Pearson Correlation (Summary Level)FineSurE
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| FineSurEDirection=LLM-based, Backbone=GPT-4, Keyfacts=Human-provided2024.07 | 0.505 | 0.451 | 0.88 | |
| FineSurEDirection=LLM-based, Backbone=GPT-4, Keyfacts=Automatically derived through LLMs2024.07 | 0.438 | 0.399 | 0.911 | |
| BERTScoreDirection=Similarity-based2024.07 | 0.416 | 0.405 | 0.783 | |
| ROUGE-1Direction=Similarity-based2024.07 | 0.387 | 0.371 | 0.332 | |
| ROUGE-2Direction=Similarity-based2024.07 | 0.328 | 0.337 | 0.29 | |
| G-EvalDirection=LLM-based, Backbone=GPT-42024.07 | 0.314 | 0.277 | 0.582 | |
| ROUGE-LDirection=Similarity-based2024.07 | 0.31 | 0.321 | 0.083 | |
| BARTScoreDirection=Similarity-based2024.07 | 0.241 | 0.21 | 0.824 | |
| UniEvalDirection=QA-based2024.07 | 0.086 | 0.128 | -0.176 |