List your company
ResearchFormerly WizWand

Multi-task Language Understanding on MMLU (PRR)

62.5PRRTAD
28.1837.094654.91Aug 20, 2024
Updated 6mo ago

Evaluation results

MethodLinks
TADBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.0862.5
SATRMDBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.0861.5
MINDBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.0860.2
MSPBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.0859.9
Mean Token EntropyBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.0858.6
PerplexityBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.0857.8
Simple FocusBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.0857
LookBackLensBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.0856.7
FocusBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.0856.3
CCPBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.0855
SARBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.0849.3
SheepsBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.0849.1
LUQBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.0844
Semantic EntropyBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.0842.7
SAPLMABackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.0842.2
Lexical Similarity Rouge-LBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.0841.8
DegMat NLI Score entail.Backbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.0840.1
Ecc. NLI Score entail.Backbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.0839.9
EVL NLI Score entail.Backbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.0839.8
EigenScoreBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.0837.1
FactoscopeBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.0836
Semantic DensityBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.0829.5