Language Understanding on MMLU (PRR)
89.3PRRTAD
Evaluation Results
| Method | Links | |
|---|---|---|
| TADBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 89.3 | |
| SATRMDBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 88.5 | |
| MINDBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 88.3 | |
| SheepsBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 88.3 | |
| LookBackLensBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 87.3 | |
| FactoscopeBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 84.4 | |
| MSPBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 65.4 | |
| Simple FocusBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 55 | |
| SAPLMABackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 41.8 | |
| CCPBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 41.2 | |
| PerplexityBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 40 | |
| LUQBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 32.1 | |
| SARBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 28.8 | |
| Lexical Similarity Rouge-LBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 27.4 | |
| EigenScoreBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 24.3 | |
| Ecc. NLI Score entail.Backbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 23.7 | |
| EVL NLI Score entail.Backbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 23.6 | |
| DegMat NLI Score entail.Backbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 23.5 | |
| Mean Token EntropyBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 22.5 | |
| Semantic DensityBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 20.3 | |
| Semantic EntropyBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 16.5 | |
| FocusBackbone=Qwen-2.5 7b, Reference Quality Metric=Acc.2024.08 | 3.7 |