Question Answering on CoQA (PRR)
0.423PRRSheeps
Evaluation Results
| Method | Links | |
|---|---|---|
| SheepsBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.423 | |
| Semantic DensityBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.401 | |
| TADBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.395 | |
| TADBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.392 | |
| Semantic DensityBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.38 | |
| SATRMDBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.359 | |
| LookBackLensBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.341 | |
| Simple FocusBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.335 | |
| SATRMDBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.333 | |
| SARBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.323 | |
| Lexical Similarity Rouge-LBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.319 | |
| LookBackLensBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.317 | |
| DegMat NLI Score entail.Backbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.316 | |
| PerplexityBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.314 | |
| EVL NLI Score entail.Backbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.314 | |
| DegMat NLI Score entail.Backbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.312 | |
| MSPBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.31 | |
| Mean Token EntropyBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.304 | |
| EVL NLI Score entail.Backbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.304 | |
| LUQBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.303 | |
| LUQBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.301 | |
| Ecc. NLI Score entail.Backbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.299 | |
| SARBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.297 | |
| Ecc. NLI Score entail.Backbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.293 | |
| FocusBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.289 | |
| Simple FocusBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.281 | |
| CCPBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.277 | |
| Lexical Similarity Rouge-LBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.277 | |
| EigenScoreBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.27 | |
| SheepsBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.27 | |
| Semantic EntropyBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.265 | |
| EigenScoreBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.263 | |
| MINDBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.263 | |
| MSPBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.262 | |
| PerplexityBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.259 | |
| MINDBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | 0.252 | |
| Mean Token EntropyBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.251 | |
| Semantic EntropyBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.232 | |
| FocusBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.211 | |
| CCPBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.21 | |
| FactoscopeBackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | 0.033 | |
| SAPLMABackbone=Llama-3.1 8b, Correctness Metric=AlignScore2024.08 | -0.002 | |
| FactoscopeBackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | -0.013 | |
| SAPLMABackbone=Gemma-2 9b, Reference Quality Metric=AlignScore2024.08 | -0.015 |