Question Answering on MedQUAD
58.3PRRTAD
Evaluation Results
| Method | Links | |
|---|---|---|
| TADBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 58.3 | |
| MINDBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 51.7 | |
| LookBackLensBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 49.7 | |
| FactoscopeBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 44.7 | |
| SheepsBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 39.1 | |
| SATRMDBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 36.2 | |
| Mean Token EntropyBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 14.3 | |
| PerplexityBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 13.1 | |
| FocusBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 12.3 | |
| EVL NLI Score entail.Backbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 12.2 | |
| DegMat NLI Score entail.Backbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 12.2 | |
| Ecc. NLI Score entail.Backbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 10.7 | |
| LUQBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 9.6 | |
| Semantic DensityBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 9.5 | |
| SARBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 8.8 | |
| Lexical Similarity Rouge-LBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 7.5 | |
| Semantic EntropyBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 7.3 | |
| MSPBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 3 | |
| Simple FocusBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 2.1 | |
| EigenScoreBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 1.8 | |
| SAPLMABackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 1.8 | |
| CCPBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 1.5 | |
| LookBackLensBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.705 | |
| MINDBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.671 | |
| SAPLMABackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.667 | |
| TADBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.624 | |
| SATRMDBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.612 | |
| TADBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.59 | |
| LookBackLensBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.547 | |
| MINDBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.531 | |
| FocusBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.514 | |
| FactoscopeBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.511 | |
| SheepsBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.501 | |
| FactoscopeBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.471 | |
| SheepsBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.436 | |
| SATRMDBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.424 | |
| Mean Token EntropyBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.422 | |
| PerplexityBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.397 | |
| Semantic DensityBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.272 | |
| PerplexityBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.249 | |
| Mean Token EntropyBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.238 | |
| SAPLMABackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.228 | |
| LUQBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.222 | |
| SARBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.203 | |
| EVL NLI Score entail.Backbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.176 | |
| Semantic DensityBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.175 | |
| Simple FocusBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.17 | |
| DegMat NLI Score entail.Backbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.167 | |
| DegMat NLI Score entail.Backbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.146 | |
| FocusBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.137 | |
| EVL NLI Score entail.Backbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.137 | |
| SARBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.107 | |
| Ecc. NLI Score entail.Backbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.102 | |
| LUQBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.101 | |
| Semantic EntropyBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.093 | |
| MSPBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.091 | |
| Semantic EntropyBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.083 | |
| Simple FocusBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.074 | |
| CCPBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.038 | |
| Ecc. NLI Score entail.Backbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.037 | |
| CCPBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.028 | |
| MSPBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.004 | |
| EigenScoreBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | -0.01 | |
| Lexical Similarity Rouge-LBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | -0.017 | |
| EigenScoreBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | -0.024 | |
| Lexical Similarity Rouge-LBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | -0.035 |