List your company
ResearchFormerly WizWand

Selective Generation on MMLU

0.947ROC-AUCTAD
0.418680.555840.6930.83016Aug 20, 2024
Updated 6mo ago

Evaluation results

MethodLinks
TADBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.080.947
SheepsBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.080.945
MINDBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.080.939
SATRMDBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.080.939
LookBackLensBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.080.928
FactoscopeBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.080.909
TADBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.080.888
MINDBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.080.884
SheepsBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.080.881
SATRMDBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.080.879
LookBackLensBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.080.874
TADBackbone=Gemma-2 9b, Reference Metric=Acc.2024.080.855
MINDBackbone=Gemma-2 9b, Reference Metric=Acc.2024.080.847
MSPBackbone=Gemma-2 9b, Reference Metric=Acc.2024.080.846
SATRMDBackbone=Gemma-2 9b, Reference Metric=Acc.2024.080.846
Mean Token EntropyBackbone=Gemma-2 9b, Reference Metric=Acc.2024.080.841
PerplexityBackbone=Gemma-2 9b, Reference Metric=Acc.2024.080.84
Simple FocusBackbone=Gemma-2 9b, Reference Metric=Acc.2024.080.838
FocusBackbone=Gemma-2 9b, Reference Metric=Acc.2024.080.83
LookBackLensBackbone=Gemma-2 9b, Reference Metric=Acc.2024.080.828
FactoscopeBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.080.82
CCPBackbone=Gemma-2 9b, Reference Metric=Acc.2024.080.816
MSPBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.080.813
SheepsBackbone=Gemma-2 9b, Reference Metric=Acc.2024.080.806
SARBackbone=Gemma-2 9b, Reference Metric=Acc.2024.080.795
Simple FocusBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.080.772
MSPBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.080.771
Semantic EntropyBackbone=Gemma-2 9b, Reference Metric=Acc.2024.080.768
Simple FocusBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.080.758
LUQBackbone=Gemma-2 9b, Reference Metric=Acc.2024.080.745
Lexical Similarity Rouge-LBackbone=Gemma-2 9b, Reference Metric=Acc.2024.080.744
PerplexityBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.080.741
Ecc. NLI Score entail.Backbone=Gemma-2 9b, Reference Metric=Acc.2024.080.739
SAPLMABackbone=Gemma-2 9b, Reference Metric=Acc.2024.080.738
EVL NLI Score entail.Backbone=Gemma-2 9b, Reference Metric=Acc.2024.080.735
SAPLMABase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.080.733
EigenScoreBackbone=Gemma-2 9b, Reference Metric=Acc.2024.080.725
DegMat NLI Score entail.Backbone=Gemma-2 9b, Reference Metric=Acc.2024.080.724
FactoscopeBackbone=Gemma-2 9b, Reference Metric=Acc.2024.080.718
SAPLMABackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.080.709
CCPBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.080.709
PerplexityBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.080.697
Semantic DensityBackbone=Gemma-2 9b, Reference Metric=Acc.2024.080.697
CCPBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.080.673
LUQBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.080.673
SARBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.080.663
SARBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.080.661
Lexical Similarity Rouge-LBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.080.654
FocusBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.080.642
EigenScoreBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.080.64
Ecc. NLI Score entail.Backbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.080.637
LUQBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.080.637
DegMat NLI Score entail.Base Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.080.633
DegMat NLI Score entail.Backbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.080.631
Lexical Similarity Rouge-LBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.080.63
EVL NLI Score entail.Base Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.080.63
EVL NLI Score entail.Backbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.080.627
Ecc. NLI Score entail.Base Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.080.625
Semantic DensityBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.080.622
Semantic DensityBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.080.621
Mean Token EntropyBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.080.619
Semantic EntropyBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.080.604
EigenScoreBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.080.603
Semantic EntropyBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.080.6
Mean Token EntropyBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.080.59
FocusBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.080.439