List your company
ResearchFormerly WizWand

Summarization on CNN (PRR)

0.444PRRTAD
-0.038560.086720.2120.33728Aug 20, 2024
Updated 6mo ago

Evaluation results

MethodLinks
TADBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.444
CCPBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.394
TADBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.383
SheepsBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.38
LookBackLensBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.367
MSPBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.348
SATRMDBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.347
CCPBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.338
MSPBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.329
SheepsBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.315
LookBackLensBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.279
Simple FocusBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.277
SATRMDBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.248
MINDBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.242
Simple FocusBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.196
FocusBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.178
MINDBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.178
LUQBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.147
LUQBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.129
Semantic DensityBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.124
EVL NLI Score entail.Backbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.113
DegMat NLI Score entail.Backbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.113
Semantic DensityBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.1
Semantic EntropyBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.085
PerplexityBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.078
PerplexityBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.076
Semantic EntropyBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.07
SAPLMABackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.068
Lexical Similarity Rouge-LBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.061
DegMat NLI Score entail.Backbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.056
Mean Token EntropyBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.056
SAPLMABackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.056
FocusBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.055
EigenScoreBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.055
SARBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.049
EVL NLI Score entail.Backbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.048
SARBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.043
Mean Token EntropyBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.038
FactoscopeBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.036
EigenScoreBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.004
Ecc. NLI Score entail.Backbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.001
Ecc. NLI Score entail.Backbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080
Lexical Similarity Rouge-LBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08-0.019
FactoscopeBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08-0.02