Question Answering on SciQ (PRR)
60PRRLookBackLens
Evaluation Results
| Method | Links | |
|---|---|---|
| LookBackLensBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 60 | |
| TADBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 56.3 | |
| MSPBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 55.1 | |
| SATRMDBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 54.7 | |
| DegMat NLI Score entail.Backbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 54 | |
| Simple FocusBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 53.6 | |
| Ecc. NLI Score entail.Backbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 53.5 | |
| SARBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 52.6 | |
| EVL NLI Score entail.Backbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 51.9 | |
| Semantic DensityBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 51.4 | |
| EigenScoreBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 51 | |
| Lexical Similarity Rouge-LBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 50.7 | |
| MINDBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 49.6 | |
| Semantic EntropyBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 49.1 | |
| SheepsBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 48.7 | |
| CCPBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 46.8 | |
| FocusBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 46.2 | |
| LUQBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 44.9 | |
| PerplexityBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 38.5 | |
| SAPLMABackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 37.4 | |
| FactoscopeBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 34.5 | |
| Mean Token EntropyBackbone=Qwen-2.5 7b, Reference Quality Metric=AlignScore2024.08 | 34.2 | |
| SATRMDBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.595 | |
| TADBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.581 | |
| SATRMDBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.581 | |
| LookBackLensBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.574 | |
| SheepsBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.552 | |
| LookBackLensBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.542 | |
| Simple FocusBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.523 | |
| MSPBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.501 | |
| MINDBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.499 | |
| TADBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.488 | |
| Simple FocusBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.486 | |
| Semantic DensityBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.463 | |
| MSPBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.459 | |
| SAPLMABackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.455 | |
| SheepsBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.455 | |
| MINDBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.454 | |
| Semantic DensityBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.448 | |
| SARBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.439 | |
| FocusBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.434 | |
| DegMat NLI Score entail.Backbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.429 | |
| DegMat NLI Score entail.Backbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.422 | |
| FocusBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.422 | |
| Ecc. NLI Score entail.Backbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.419 | |
| SAPLMABackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.399 | |
| Lexical Similarity Rouge-LBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.395 | |
| LUQBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.394 | |
| EVL NLI Score entail.Backbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.389 | |
| CCPBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.385 | |
| Ecc. NLI Score entail.Backbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.38 | |
| Lexical Similarity Rouge-LBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.378 | |
| EVL NLI Score entail.Backbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.371 | |
| SARBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.362 | |
| EigenScoreBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.359 | |
| Semantic EntropyBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.355 | |
| EigenScoreBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.355 | |
| CCPBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.351 | |
| Semantic EntropyBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.347 | |
| LUQBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.342 | |
| FactoscopeBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.313 | |
| FactoscopeBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.306 | |
| PerplexityBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.244 | |
| PerplexityBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.234 | |
| Mean Token EntropyBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.234 | |
| Mean Token EntropyBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.218 |