List your company
ResearchFormerly WizWand

Selective Generation on TriviaQA

87.4ROC-AUCMean Token Entropy
63.06469.38275.782.018Aug 20, 2024
Updated 6mo ago

Evaluation results

MethodLinks
Mean Token EntropyBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0887.4
TADBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0887.3
SATRMDBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0886.9
PerplexityBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0886.7
Simple FocusBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0886.5
MSPBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0886.3
TADBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0885.9
Semantic DensityBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0885.8
CCPBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0885.7
TADBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0885.4
SARBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0885.1
SheepsBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0884.6
DegMat NLI Score entail.Backbone=Gemma-2 9b, Reference Metric=AlignScore2024.0884.5
EVL NLI Score entail.Backbone=Gemma-2 9b, Reference Metric=AlignScore2024.0884.4
LUQBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0884.4
FocusBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0884.2
SATRMDBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0883.6
DegMat NLI Score entail.Base Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0883.4
Mean Token EntropyBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0883.2
Lexical Similarity Rouge-LBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0883
Semantic DensityBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0882.9
Semantic DensityBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0882.8
EVL NLI Score entail.Base Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0882.8
LookBackLensBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0882.7
SheepsBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0882.7
Simple FocusBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0882.6
SARBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0882.5
CCPBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0882.4
LUQBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0882.3
PerplexityBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0882.2
CCPBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0882.1
SARBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0882.1
Simple FocusBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0882.1
Semantic EntropyBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0882
MSPBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0881.9
DegMat NLI Score entail.Backbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0881.9
SATRMDBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0881.9
Ecc. NLI Score entail.Backbone=Gemma-2 9b, Reference Metric=AlignScore2024.0881.8
LookBackLensBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0881.6
Mean Token EntropyBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0881.5
EVL NLI Score entail.Backbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0881.4
LUQBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0881.4
MINDBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0881.2
MSPBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0880.9
Ecc. NLI Score entail.Base Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0880.9
PerplexityBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0880.4
FocusBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0879.9
Lexical Similarity Rouge-LBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0879.9
Ecc. NLI Score entail.Backbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0879.7
EigenScoreBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0879.7
Lexical Similarity Rouge-LBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0879.6
Semantic EntropyBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0879.5
FocusBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0879.3
Semantic EntropyBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0879.2
MINDBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0878.5
SheepsBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0878.2
EigenScoreBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0878
LookBackLensBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0877.8
SAPLMABackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0877.6
EigenScoreBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0877.5
MINDBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0875.6
SAPLMABackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0874.9
SAPLMABase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0872.8
FactoscopeBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0871.6
FactoscopeBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0870.5
FactoscopeBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0864