List your company
ResearchFormerly WizWand

Selective Generation on CoQA (ROC-AUC)

74.7ROC-AUCTAD
48.28455.1426268.858Aug 20, 2024
Updated 6mo ago

Evaluation results

MethodLinks
TADBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0874.7
SheepsBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0874.6
TADBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0874.4
Semantic DensityBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0873.4
Semantic DensityBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0873.1
TADBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0873
MINDBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0872.2
Semantic DensityBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0872
SATRMDBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0871.8
Simple FocusBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0871.2
LookBackLensBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0870.6
EVL NLI Score entail.Backbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0870.3
PerplexityBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0870.3
SATRMDBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0870.2
LookBackLensBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0870.1
SARBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0870
LUQBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0869.9
Simple FocusBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0869.8
DegMat NLI Score entail.Backbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0869.8
MSPBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0869.8
SATRMDBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0869.7
EVL NLI Score entail.Backbone=Gemma-2 9b, Reference Metric=AlignScore2024.0869.7
LookBackLensBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0869.7
SARBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0869.5
SheepsBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0869.5
Ecc. NLI Score entail.Backbone=Gemma-2 9b, Reference Metric=AlignScore2024.0869.4
DegMat NLI Score entail.Backbone=Gemma-2 9b, Reference Metric=AlignScore2024.0869.4
Lexical Similarity Rouge-LBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0869.3
EVL NLI Score entail.Base Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0869.1
Mean Token EntropyBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0868.9
DegMat NLI Score entail.Base Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0868.9
Ecc. NLI Score entail.Backbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0868.5
LUQBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0868.4
LUQBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0868.4
Lexical Similarity Rouge-LBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0868.3
SARBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0868.3
Semantic EntropyBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0867.9
MSPBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0867.8
Ecc. NLI Score entail.Base Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0867.7
FocusBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0867.6
Lexical Similarity Rouge-LBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0867.5
PerplexityBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0867.4
Semantic EntropyBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0867.3
EigenScoreBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0867.3
Simple FocusBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0867.1
EigenScoreBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0867
CCPBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0866.9
Mean Token EntropyBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0866.8
SheepsBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0866.8
PerplexityBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0866.5
Mean Token EntropyBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0866.2
MINDBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0866
EigenScoreBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0865.8
MSPBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0865.5
MINDBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0865.4
Semantic EntropyBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0865.3
FocusBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0864.1
CCPBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0863.9
CCPBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0863.3
FocusBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0862.4
FactoscopeBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0858.5
FactoscopeBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0851.3
SAPLMABase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.0850.9
SAPLMABackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.0850.1
SAPLMABackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0849.9
FactoscopeBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.0849.3