Summarization on CNN (PRR)
0.444PRRTAD
Evaluation Results
| Method | Links | |
|---|---|---|
| TADBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.444 | |
| CCPBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.394 | |
| TADBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.383 | |
| SheepsBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.38 | |
| LookBackLensBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.367 | |
| MSPBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.348 | |
| SATRMDBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.347 | |
| CCPBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.338 | |
| MSPBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.329 | |
| SheepsBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.315 | |
| LookBackLensBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.279 | |
| Simple FocusBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.277 | |
| SATRMDBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.248 | |
| MINDBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.242 | |
| Simple FocusBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.196 | |
| FocusBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.178 | |
| MINDBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.178 | |
| LUQBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.147 | |
| LUQBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.129 | |
| Semantic DensityBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.124 | |
| EVL NLI Score entail.Backbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.113 | |
| DegMat NLI Score entail.Backbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.113 | |
| Semantic DensityBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.1 | |
| Semantic EntropyBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.085 | |
| PerplexityBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.078 | |
| PerplexityBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.076 | |
| Semantic EntropyBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.07 | |
| SAPLMABackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.068 | |
| Lexical Similarity Rouge-LBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.061 | |
| DegMat NLI Score entail.Backbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.056 | |
| Mean Token EntropyBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.056 | |
| SAPLMABackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.056 | |
| FocusBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.055 | |
| EigenScoreBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.055 | |
| SARBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.049 | |
| EVL NLI Score entail.Backbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.048 | |
| SARBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.043 | |
| Mean Token EntropyBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.038 | |
| FactoscopeBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.036 | |
| EigenScoreBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.004 | |
| Ecc. NLI Score entail.Backbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.001 | |
| Ecc. NLI Score entail.Backbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0 | |
| Lexical Similarity Rouge-LBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | -0.019 | |
| FactoscopeBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | -0.02 |