List your company
ResearchFormerly WizWand

Selective Generation on MedQUAD

0.928ROC-AUCMIND
0.460.58150.7030.8245Aug 20, 2024
Updated 6mo ago

Evaluation results

MethodLinks
MINDBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.928
FactoscopeBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.925
SATRMDBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.92
LookBackLensBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.915
TADBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.896
SheepsBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.889
SAPLMABackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.868
MINDBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.868
FocusBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.852
LookBackLensBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.846
Mean Token EntropyBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.839
PerplexityBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.837
FactoscopeBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.832
TADBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.829
SheepsBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.819
CCPBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.81
Simple FocusBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.805
MSPBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.798
TADBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.788
SATRMDBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.767
SAPLMABase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.76
LookBackLensBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.75
FactoscopeBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.735
FocusBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.731
Mean Token EntropyBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.72
PerplexityBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.718
MSPBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.717
MINDBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.714
SATRMDBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.714
CCPBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.713
Simple FocusBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.711
SheepsBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.707
PerplexityBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.684
Mean Token EntropyBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.679
Semantic DensityBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.679
MSPBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.67
CCPBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.665
Semantic DensityBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.655
Semantic EntropyBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.645
SARBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.641
SARBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.641
FocusBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.626
Semantic EntropyBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.605
Simple FocusBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.604
Semantic DensityBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.602
SARBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.597
Semantic EntropyBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.588
DegMat NLI Score entail.Base Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.572
EVL NLI Score entail.Base Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.565
Ecc. NLI Score entail.Base Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.565
Ecc. NLI Score entail.Backbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.554
LUQBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.554
DegMat NLI Score entail.Backbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.544
EigenScoreBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.543
EVL NLI Score entail.Backbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.536
Lexical Similarity Rouge-LBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.080.53
Ecc. NLI Score entail.Backbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.528
EigenScoreBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.526
LUQBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.521
Lexical Similarity Rouge-LBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.519
DegMat NLI Score entail.Backbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.506
EVL NLI Score entail.Backbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.497
Lexical Similarity Rouge-LBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.495
LUQBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.479
SAPLMABackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.080.478
EigenScoreBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.080.478