List your company
ResearchFormerly WizWand

Mathematical Reasoning on GSM8k (PRR)

0.738PRRMIND
-0.134560.091970.31850.54503Aug 20, 2024
Updated 6mo ago

Evaluation results

MethodLinks
MINDBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.080.738
SheepsBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.080.71
TADBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.080.701
SheepsBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.080.654
LookBackLensBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.080.627
TADBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.080.61
SheepsBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.080.604
MINDBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.080.592
MINDBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.080.57
SATRMDBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.080.566
LookBackLensBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.080.562
TADBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.080.557
EigenScoreBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.080.537
SATRMDBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.080.528
LookBackLensBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.080.525
Lexical Similarity Rouge-LBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.080.511
Mean Token EntropyBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.080.469
SARBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.080.459
PerplexityBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.080.456
SAPLMABackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.080.447
SATRMDBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.080.443
SAPLMABackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.080.44
DegMat NLI Score entail.Backbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.080.402
Ecc. NLI Score entail.Backbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.080.386
Semantic DensityBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.080.381
Semantic EntropyBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.080.38
EVL NLI Score entail.Backbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.080.372
Semantic EntropyBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.080.366
SAPLMABackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.080.358
SARBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.080.355
Lexical Similarity Rouge-LBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.080.346
CCPBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.080.339
Mean Token EntropyBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.080.333
LUQBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.080.331
Semantic EntropyBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.080.328
SARBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.080.32
Simple FocusBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.080.31
MSPBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.080.31
MSPBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.080.31
CCPBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.080.306
Mean Token EntropyBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.080.304
PerplexityBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.080.303
Simple FocusBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.080.302
DegMat NLI Score entail.Backbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.080.293
EVL NLI Score entail.Backbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.080.284
EigenScoreBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.080.283
CCPBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.080.281
Simple FocusBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.080.28
FocusBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.080.278
FocusBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.080.273
Lexical Similarity Rouge-LBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.080.273
MSPBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.080.268
FocusBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.080.265
PerplexityBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.080.256
EigenScoreBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.080.241
LUQBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.080.237
Ecc. NLI Score entail.Backbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.080.235
Ecc. NLI Score entail.Backbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.080.228
DegMat NLI Score entail.Backbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.080.203
Semantic DensityBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.080.197
EVL NLI Score entail.Backbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.080.188
Semantic DensityBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.080.183
LUQBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.080.158
FactoscopeBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.080.121
FactoscopeBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.080.028
FactoscopeBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08-0.101