List your company
ResearchFormerly WizWand

Summarization on SamSum (PRR)

-0.113PRRFactoscope
-2.2575212.2179926.693541.16901Aug 20, 2024
Updated 6mo ago

Evaluation results

MethodLinks
FactoscopeBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08-0.113
FactoscopeBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08-0.032
EigenScoreBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08-0.01
MSPBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.012
CCPBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.025
Ecc. NLI Score entail.Backbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.032
Lexical Similarity Rouge-LBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.057
Ecc. NLI Score entail.Backbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.069
Semantic EntropyBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.074
EigenScoreBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.079
CCPBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.108
Semantic EntropyBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.114
Simple FocusBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.116
Semantic DensityBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.122
MSPBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.125
SARBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.129
MINDBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.133
FocusBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.144
EVL NLI Score entail.Backbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.145
DegMat NLI Score entail.Backbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.145
LUQBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.17
Simple FocusBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.174
Lexical Similarity Rouge-LBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.186
Semantic DensityBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.198
LUQBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.207
SARBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.215
PerplexityBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.25
EVL NLI Score entail.Backbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.261
PerplexityBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.263
DegMat NLI Score entail.Backbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.263
MINDBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.274
Mean Token EntropyBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.278
Mean Token EntropyBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.28
SheepsBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.313
SATRMDBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.322
FocusBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.323
SAPLMABackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.326
SheepsBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.327
SATRMDBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.33
LookBackLensBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.386
Ecc. NLI Score entail.Backbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.080.4
SAPLMABackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.406
TADBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.080.416
TADBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.498
LookBackLensBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.080.501
FactoscopeBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.086.4
CCPBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.086.5
EigenScoreBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.087
Semantic EntropyBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.088.3
EVL NLI Score entail.Backbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.088.6
DegMat NLI Score entail.Backbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.088.9
MSPBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.0810.7
Semantic DensityBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.0812.2
Lexical Similarity Rouge-LBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.0813.2
SARBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.0816.6
LUQBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.0821
Simple FocusBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.0829
FocusBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.0832.4
MINDBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.0836.1
SAPLMABackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.0838.2
PerplexityBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.0840.8
Mean Token EntropyBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.0842.4
LookBackLensBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.0844.1
SheepsBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.0846.6
SATRMDBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.0849.5
TADBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.0853.5