List your company
ResearchFormerly WizWand

Selective Generation on WMT19

0.845ROC-AUCLookBackLens
0.516360.601680.6870.77232Aug 20, 2024
Updated 6mo ago

Evaluation results

MethodLinks
LookBackLensBackbone=Gemma-2 9b, Reference Metric=Comet2024.080.845
SAPLMABackbone=Gemma-2 9b, Reference Metric=Comet2024.080.831
SheepsBackbone=Gemma-2 9b, Reference Metric=Comet2024.080.831
MINDBackbone=Gemma-2 9b, Reference Metric=Comet2024.080.826
TADBackbone=Gemma-2 9b, Reference Metric=Comet2024.080.824
LookBackLensBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.080.82
TADBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.080.812
TADBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.080.795
SAPLMABase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.080.792
SheepsBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.080.778
SATRMDBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.080.774
SATRMDBackbone=Gemma-2 9b, Reference Metric=Comet2024.080.774
LookBackLensBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.080.772
Simple FocusBackbone=Gemma-2 9b, Reference Metric=Comet2024.080.761
Mean Token EntropyBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.080.76
SheepsBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.080.76
PerplexityBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.080.755
Simple FocusBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.080.751
MINDBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.080.748
SARBackbone=Gemma-2 9b, Reference Metric=Comet2024.080.745
SARBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.080.743
Simple FocusBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.080.738
Mean Token EntropyBackbone=Gemma-2 9b, Reference Metric=Comet2024.080.734
MSPBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.080.733
PerplexityBackbone=Gemma-2 9b, Reference Metric=Comet2024.080.733
MSPBackbone=Gemma-2 9b, Reference Metric=Comet2024.080.732
SATRMDBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.080.729
Mean Token EntropyBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.080.728
SARBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.080.727
MSPBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.080.726
FocusBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.080.725
PerplexityBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.080.722
FocusBackbone=Gemma-2 9b, Reference Metric=Comet2024.080.721
Lexical Similarity Rouge-LBackbone=Gemma-2 9b, Reference Metric=Comet2024.080.713
Lexical Similarity Rouge-LBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.080.709
FocusBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.080.708
Semantic EntropyBackbone=Gemma-2 9b, Reference Metric=Comet2024.080.706
Semantic EntropyBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.080.703
Lexical Similarity Rouge-LBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.080.703
EigenScoreBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.080.702
Semantic EntropyBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.080.692
CCPBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.080.688
Ecc. NLI Score entail.Backbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.080.688
MINDBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.080.682
Ecc. NLI Score entail.Backbone=Gemma-2 9b, Reference Metric=Comet2024.080.681
CCPBackbone=Gemma-2 9b, Reference Metric=Comet2024.080.679
Semantic DensityBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.080.677
DegMat NLI Score entail.Backbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.080.675
DegMat NLI Score entail.Backbone=Gemma-2 9b, Reference Metric=Comet2024.080.671
CCPBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.080.671
EVL NLI Score entail.Backbone=Gemma-2 9b, Reference Metric=Comet2024.080.668
EigenScoreBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.080.668
SAPLMABackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.080.667
EVL NLI Score entail.Backbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.080.665
Semantic DensityBackbone=Gemma-2 9b, Reference Metric=Comet2024.080.661
Ecc. NLI Score entail.Base Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.080.654
LUQBackbone=Gemma-2 9b, Reference Metric=Comet2024.080.648
LUQBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.080.647
DegMat NLI Score entail.Base Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.080.64
EigenScoreBackbone=Gemma-2 9b, Reference Metric=Comet2024.080.639
Semantic DensityBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.080.638
EVL NLI Score entail.Base Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.080.63
LUQBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.080.629
FactoscopeBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.080.592
FactoscopeBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.080.54
FactoscopeBackbone=Gemma-2 9b, Reference Metric=Comet2024.080.529