List your company
ResearchFormerly WizWand

Selective Generation on TruthfulQA

0.744ROC-AUCTAD
0.440320.519160.5980.67684Aug 20, 2024
Updated 6mo ago

Evaluation results

MethodLinks
TADBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.744
LookBackLensBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.739
TADBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.738
SheepsBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.736
MINDBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.729
TADBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.726
SAPLMABackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.725
LookBackLensBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.718
LookBackLensBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.716
SheepsBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.703
MINDBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.696
SATRMDBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.694
SheepsBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.687
SAPLMABackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.68
MINDBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.68
Semantic DensityBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.68
SATRMDBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.679
LUQBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.663
LUQBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.657
Mean Token EntropyBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.652
EVL NLI Score entail.Backbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.648
Mean Token EntropyBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.648
DegMat NLI Score entail.Backbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.642
PerplexityBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.641
FocusBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.641
Semantic DensityBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.634
SATRMDBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.622
Mean Token EntropyBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.622
SARBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.621
FocusBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.619
PerplexityBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.617
PerplexityBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.616
SARBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.615
EVL NLI Score entail.Backbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.614
Simple FocusBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.613
MSPBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.612
Semantic DensityBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.612
DegMat NLI Score entail.Backbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.608
DegMat NLI Score entail.Base Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.606
EVL NLI Score entail.Base Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.605
SAPLMABase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.603
FocusBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.597
Simple FocusBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.588
Lexical Similarity Rouge-LBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.583
LUQBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.58
Ecc. NLI Score entail.Backbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.579
Simple FocusBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.565
Ecc. NLI Score entail.Backbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.565
SARBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.564
Ecc. NLI Score entail.Base Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.559
Lexical Similarity Rouge-LBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.557
CCPBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.556
MSPBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.55
EigenScoreBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.55
FactoscopeBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.546
FactoscopeBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.541
Semantic EntropyBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.538
FactoscopeBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.537
EigenScoreBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.537
Semantic EntropyBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.529
Lexical Similarity Rouge-LBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.525
CCPBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.52
EigenScoreBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.502
Semantic EntropyBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.496
MSPBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.463
CCPBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.452