List your company
ResearchFormerly WizWand

Summarization on XSum (PRR)

0.617PRRTAD
-0.073560.105720.2850.46428Aug 20, 2024
Updated 6mo ago

Evaluation results

MethodLinks
TADBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.617
LookBackLensBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.593
TADBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.55
LookBackLensBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.528
SheepsBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.523
SheepsBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.51
SATRMDBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.506
SATRMDBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.494
TADBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.46
MINDBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.437
LookBackLensBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.436
MINDBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.428
CCPBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.399
SAPLMABackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.395
SheepsBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.361
PerplexityBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.353
Mean Token EntropyBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.342
CCPBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.339
SATRMDBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.338
PerplexityBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.325
MSPBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.321
Mean Token EntropyBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.312
FocusBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.31
MSPBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.303
FactoscopeBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.292
SAPLMABackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.288
FocusBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.266
Simple FocusBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.254
Simple FocusBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.248
SAPLMABackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.245
CCPBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.24
PerplexityBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.237
FactoscopeBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.234
Mean Token EntropyBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.233
LUQBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.228
MINDBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.22
FocusBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.209
LUQBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.197
FactoscopeBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.185
Semantic DensityBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.172
Semantic DensityBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.163
EVL NLI Score entail.Backbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.139
DegMat NLI Score entail.Backbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.138
LUQBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.137
SARBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.128
Lexical Similarity Rouge-LBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.122
Simple FocusBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.109
SARBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.086
Semantic DensityBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.08
MSPBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.077
EigenScoreBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.077
Lexical Similarity Rouge-LBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.076
Lexical Similarity Rouge-LBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.059
EVL NLI Score entail.Backbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.058
DegMat NLI Score entail.Backbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.058
SARBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.052
EigenScoreBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.033
EVL NLI Score entail.Backbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.033
DegMat NLI Score entail.Backbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.031
EigenScoreBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.024
Semantic EntropyBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.016
Semantic EntropyBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.016
Ecc. NLI Score entail.Backbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.012
Semantic EntropyBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.011
Ecc. NLI Score entail.Backbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.004
Ecc. NLI Score entail.Backbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08-0.047