List your company
ResearchFormerly WizWand

Selective Generation on XSum

85.9ROC-AUCTAD
45.44455.94766.4576.953Aug 20, 2024
Updated 6mo ago

Evaluation results

MethodLinks
TADBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0885.9
TADBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0885.1
LookBackLensBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0884.7
LookBackLensBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0883.5
SheepsBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0882.4
SATRMDBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0880.7
SATRMDBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0880.2
MINDBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0878.6
SheepsBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0877.6
TADBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0877.4
LookBackLensBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0877.2
MINDBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0876.5
SAPLMABackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0875.5
PerplexityBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0875
CCPBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0874.8
Mean Token EntropyBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0874.3
PerplexityBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0873
CCPBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0872.7
Mean Token EntropyBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0872.5
SheepsBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0872.4
MSPBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0871.2
FocusBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0871
FactoscopeBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0870.7
MSPBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0870.5
SATRMDBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0870
SAPLMABase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0869.8
Simple FocusBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0868.3
FocusBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0867.8
FactoscopeBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0867.7
LUQBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0866.8
SAPLMABackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0866.6
Simple FocusBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0866.6
PerplexityBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0865.3
MINDBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0865.1
Mean Token EntropyBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0865
CCPBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0865
FocusBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0864.4
LUQBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0863.1
FactoscopeBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0861.7
EVL NLI Score entail.Backbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0861.1
DegMat NLI Score entail.Backbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0861.1
SARBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0858.9
Semantic DensityBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0858.7
LUQBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0858.7
Semantic DensityBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0858.6
Lexical Similarity Rouge-LBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0857.9
SARBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0857.4
Simple FocusBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0857.1
SARBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0854.8
EVL NLI Score entail.Backbone=Gemma-2 9b, Reference Metric=AlignScore2024.0854.4
Lexical Similarity Rouge-LBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0854.3
DegMat NLI Score entail.Backbone=Gemma-2 9b, Reference Metric=AlignScore2024.0854.3
EigenScoreBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0854.1
Lexical Similarity Rouge-LBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0853.9
Semantic DensityBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0853
MSPBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0852.4
EigenScoreBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0852.1
EigenScoreBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0852
Ecc. NLI Score entail.Backbone=Gemma-2 9b, Reference Metric=AlignScore2024.0851.2
DegMat NLI Score entail.Base Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0850.9
EVL NLI Score entail.Base Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0850.8
Semantic EntropyBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0850.6
Semantic EntropyBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0850.1
Semantic EntropyBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0849.1
Ecc. NLI Score entail.Backbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0847.9
Ecc. NLI Score entail.Base Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0847