List your company
ResearchFormerly WizWand

Selective Generation on GSM8k

88.5ROC-AUCSheeps
38.99651.84864.777.552Aug 20, 2024
Updated 6mo ago

Evaluation results

MethodLinks
SheepsBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.0888.5
MINDBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.0888.2
TADBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.0888.1
SheepsBackbone=Gemma-2 9b, Reference Metric=Acc.2024.0885.3
TADBackbone=Gemma-2 9b, Reference Metric=Acc.2024.0884.4
LookBackLensBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.0883.9
MINDBackbone=Gemma-2 9b, Reference Metric=Acc.2024.0882.1
SATRMDBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.0881.9
LookBackLensBackbone=Gemma-2 9b, Reference Metric=Acc.2024.0881.7
SheepsBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.0881.6
EigenScoreBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.0881.2
TADBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.0880.9
Lexical Similarity Rouge-LBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.0879.5
MINDBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.0879.5
SATRMDBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.0878.7
LookBackLensBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.0878
SARBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.0877.3
SAPLMABackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.0876.1
SAPLMABackbone=Gemma-2 9b, Reference Metric=Acc.2024.0876
Mean Token EntropyBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.0875.6
SATRMDBackbone=Gemma-2 9b, Reference Metric=Acc.2024.0875.5
PerplexityBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.0874.6
DegMat NLI Score entail.Backbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.0874.3
Semantic DensityBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.0873.9
EVL NLI Score entail.Backbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.0873.3
Semantic EntropyBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.0873.1
Ecc. NLI Score entail.Backbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.0873
SARBackbone=Gemma-2 9b, Reference Metric=Acc.2024.0872.6
Lexical Similarity Rouge-LBackbone=Gemma-2 9b, Reference Metric=Acc.2024.0872.2
LUQBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.0872.1
SAPLMABase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.0871.3
Semantic EntropyBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.0870.3
Semantic EntropyBackbone=Gemma-2 9b, Reference Metric=Acc.2024.0870.2
CCPBackbone=Gemma-2 9b, Reference Metric=Acc.2024.0869.9
Simple FocusBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.0869.3
DegMat NLI Score entail.Backbone=Gemma-2 9b, Reference Metric=Acc.2024.0869.1
SARBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.0868.9
EVL NLI Score entail.Backbone=Gemma-2 9b, Reference Metric=Acc.2024.0868.5
Lexical Similarity Rouge-LBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.0868.2
MSPBackbone=Gemma-2 9b, Reference Metric=Acc.2024.0868.1
CCPBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.0867.9
CCPBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.0867.8
EigenScoreBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.0867.7
FocusBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.0867.6
Ecc. NLI Score entail.Backbone=Gemma-2 9b, Reference Metric=Acc.2024.0867.3
MSPBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.0867.2
Simple FocusBackbone=Gemma-2 9b, Reference Metric=Acc.2024.0867.1
MSPBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.0867
Simple FocusBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.0866.9
FocusBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.0866.8
FocusBackbone=Gemma-2 9b, Reference Metric=Acc.2024.0866.4
Mean Token EntropyBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.0866.4
EigenScoreBackbone=Gemma-2 9b, Reference Metric=Acc.2024.0866.2
LUQBackbone=Gemma-2 9b, Reference Metric=Acc.2024.0866.1
Mean Token EntropyBackbone=Gemma-2 9b, Reference Metric=Acc.2024.0865.7
Ecc. NLI Score entail.Base Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.0865.5
PerplexityBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.0865.2
DegMat NLI Score entail.Base Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.0863.7
PerplexityBackbone=Gemma-2 9b, Reference Metric=Acc.2024.0863.4
Semantic DensityBackbone=Gemma-2 9b, Reference Metric=Acc.2024.0863.4
EVL NLI Score entail.Base Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.0862.7
Semantic DensityBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.0861.4
LUQBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.0860.5
FactoscopeBase Model=Llama-3.1 8b, Ground Truth Metric=Acc.2024.0855.8
FactoscopeBackbone=Gemma-2 9b, Reference Metric=Acc.2024.0852.3
FactoscopeBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Acc.2024.0840.9