Multi-task Language Understanding on MMLU (PRR)
62.5PRRTAD
Evaluation Results
| Method | Links | |
|---|---|---|
| TADBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 62.5 | |
| SATRMDBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 61.5 | |
| MINDBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 60.2 | |
| MSPBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 59.9 | |
| Mean Token EntropyBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 58.6 | |
| PerplexityBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 57.8 | |
| Simple FocusBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 57 | |
| LookBackLensBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 56.7 | |
| FocusBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 56.3 | |
| CCPBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 55 | |
| SARBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 49.3 | |
| SheepsBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 49.1 | |
| LUQBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 44 | |
| Semantic EntropyBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 42.7 | |
| SAPLMABackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 42.2 | |
| Lexical Similarity Rouge-LBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 41.8 | |
| DegMat NLI Score entail.Backbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 40.1 | |
| Ecc. NLI Score entail.Backbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 39.9 | |
| EVL NLI Score entail.Backbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 39.8 | |
| EigenScoreBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 37.1 | |
| FactoscopeBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 36 | |
| Semantic DensityBackbone=Gemma-2 9b, Reference Quality Metric=Acc.2024.08 | 29.5 |