Mathematical Reasoning on GSM8k (PRR)
0.738PRRMIND
Evaluation Results
| Method | Links | |
|---|---|---|
| MINDBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 0.738 | |
| SheepsBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 0.71 | |
| TADBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 0.701 | |
| SheepsBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 0.654 | |
| LookBackLensBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 0.627 | |
| TADBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 0.61 | |
| SheepsBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.08 | 0.604 | |
| MINDBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 0.592 | |
| MINDBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.08 | 0.57 | |
| SATRMDBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 0.566 | |
| LookBackLensBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 0.562 | |
| TADBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.08 | 0.557 | |
| EigenScoreBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 0.537 | |
| SATRMDBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.08 | 0.528 | |
| LookBackLensBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.08 | 0.525 | |
| Lexical Similarity Rouge-LBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 0.511 | |
| Mean Token EntropyBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 0.469 | |
| SARBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 0.459 | |
| PerplexityBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 0.456 | |
| SAPLMABackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 0.447 | |
| SATRMDBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 0.443 | |
| SAPLMABackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 0.44 | |
| DegMat NLI Score entail.Backbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 0.402 | |
| Ecc. NLI Score entail.Backbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 0.386 | |
| Semantic DensityBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 0.381 | |
| Semantic EntropyBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 0.38 | |
| EVL NLI Score entail.Backbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 0.372 | |
| Semantic EntropyBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.08 | 0.366 | |
| SAPLMABackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.08 | 0.358 | |
| SARBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 0.355 | |
| Lexical Similarity Rouge-LBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 0.346 | |
| CCPBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 0.339 | |
| Mean Token EntropyBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.08 | 0.333 | |
| LUQBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 0.331 | |
| Semantic EntropyBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 0.328 | |
| SARBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.08 | 0.32 | |
| Simple FocusBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 0.31 | |
| MSPBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 0.31 | |
| MSPBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.08 | 0.31 | |
| CCPBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.08 | 0.306 | |
| Mean Token EntropyBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 0.304 | |
| PerplexityBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.08 | 0.303 | |
| Simple FocusBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.08 | 0.302 | |
| DegMat NLI Score entail.Backbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 0.293 | |
| EVL NLI Score entail.Backbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 0.284 | |
| EigenScoreBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.08 | 0.283 | |
| CCPBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 0.281 | |
| Simple FocusBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 0.28 | |
| FocusBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.08 | 0.278 | |
| FocusBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 0.273 | |
| Lexical Similarity Rouge-LBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.08 | 0.273 | |
| MSPBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 0.268 | |
| FocusBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 0.265 | |
| PerplexityBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 0.256 | |
| EigenScoreBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 0.241 | |
| LUQBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 0.237 | |
| Ecc. NLI Score entail.Backbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.08 | 0.235 | |
| Ecc. NLI Score entail.Backbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 0.228 | |
| DegMat NLI Score entail.Backbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.08 | 0.203 | |
| Semantic DensityBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.08 | 0.197 | |
| EVL NLI Score entail.Backbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.08 | 0.188 | |
| Semantic DensityBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 0.183 | |
| LUQBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.08 | 0.158 | |
| FactoscopeBackbone=Llama-3.1 8b, Correctness Metric=Accuracy2024.08 | 0.121 | |
| FactoscopeBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 0.028 | |
| FactoscopeBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | -0.101 |