Summary Completeness Evaluation on REALSumm
0.688Summary Pearson RFineSurE (GPT-4)
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| FineSurE (GPT-4)Direction=LLM-based, Backbone=GPT-4, Keyfact Source=Human2024.07 | 0.688 | 0.677 | 0.949 | |
| FineSurE† (GPT-4)Direction=LLM-based, Backbone=GPT-4, Keyfact Source=Automated (LLM-derived)2024.07 | 0.571 | 0.546 | 0.905 | |
| ROUGE-1Direction=Similarity based2024.07 | 0.484 | 0.461 | 0.516 | |
| ROUGE-2Direction=Similarity based2024.07 | 0.456 | 0.461 | 0.463 | |
| BERTScoreDirection=Similarity based2024.07 | 0.455 | 0.443 | 0.619 | |
| ROUGE-LDirection=Similarity based2024.07 | 0.425 | 0.428 | 0.238 | |
| G-Eval (GPT4)Direction=LLM-based, Backbone=GPT-42024.07 | 0.314 | 0.295 | 0.908 | |
| BARTScoreDirection=Similarity based2024.07 | 0.216 | 0.199 | 0.653 | |
| UniEvalDirection=QA-based2024.07 | 0.134 | 0.18 | 0.346 |