Selective Generation on MedQUAD
0.928ROC-AUCMIND
Evaluation Results
| Method | Links | |
|---|---|---|
| MINDBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 0.928 | |
| FactoscopeBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 0.925 | |
| SATRMDBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 0.92 | |
| LookBackLensBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 0.915 | |
| TADBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 0.896 | |
| SheepsBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 0.889 | |
| SAPLMABackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 0.868 | |
| MINDBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 0.868 | |
| FocusBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 0.852 | |
| LookBackLensBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 0.846 | |
| Mean Token EntropyBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 0.839 | |
| PerplexityBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 0.837 | |
| FactoscopeBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 0.832 | |
| TADBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 0.829 | |
| SheepsBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 0.819 | |
| CCPBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 0.81 | |
| Simple FocusBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 0.805 | |
| MSPBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 0.798 | |
| TADBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 0.788 | |
| SATRMDBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 0.767 | |
| SAPLMABase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 0.76 | |
| LookBackLensBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 0.75 | |
| FactoscopeBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 0.735 | |
| FocusBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 0.731 | |
| Mean Token EntropyBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 0.72 | |
| PerplexityBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 0.718 | |
| MSPBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 0.717 | |
| MINDBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 0.714 | |
| SATRMDBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 0.714 | |
| CCPBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 0.713 | |
| Simple FocusBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 0.711 | |
| SheepsBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 0.707 | |
| PerplexityBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 0.684 | |
| Mean Token EntropyBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 0.679 | |
| Semantic DensityBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 0.679 | |
| MSPBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 0.67 | |
| CCPBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 0.665 | |
| Semantic DensityBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 0.655 | |
| Semantic EntropyBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 0.645 | |
| SARBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 0.641 | |
| SARBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 0.641 | |
| FocusBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 0.626 | |
| Semantic EntropyBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 0.605 | |
| Simple FocusBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 0.604 | |
| Semantic DensityBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 0.602 | |
| SARBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 0.597 | |
| Semantic EntropyBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 0.588 | |
| DegMat NLI Score entail.Base Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 0.572 | |
| EVL NLI Score entail.Base Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 0.565 | |
| Ecc. NLI Score entail.Base Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 0.565 | |
| Ecc. NLI Score entail.Backbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 0.554 | |
| LUQBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 0.554 | |
| DegMat NLI Score entail.Backbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 0.544 | |
| EigenScoreBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 0.543 | |
| EVL NLI Score entail.Backbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 0.536 | |
| Lexical Similarity Rouge-LBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 0.53 | |
| Ecc. NLI Score entail.Backbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 0.528 | |
| EigenScoreBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 0.526 | |
| LUQBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 0.521 | |
| Lexical Similarity Rouge-LBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 0.519 | |
| DegMat NLI Score entail.Backbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 0.506 | |
| EVL NLI Score entail.Backbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 0.497 | |
| Lexical Similarity Rouge-LBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 0.495 | |
| LUQBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 0.479 | |
| SAPLMABackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 0.478 | |
| EigenScoreBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 0.478 |