Selective Generation on TriviaQA
87.4ROC-AUCMean Token Entropy
Evaluation Results
| Method | Links | |
|---|---|---|
| Mean Token EntropyBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 87.4 | |
| TADBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 87.3 | |
| SATRMDBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 86.9 | |
| PerplexityBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 86.7 | |
| Simple FocusBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 86.5 | |
| MSPBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 86.3 | |
| TADBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 85.9 | |
| Semantic DensityBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 85.8 | |
| CCPBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 85.7 | |
| TADBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 85.4 | |
| SARBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 85.1 | |
| SheepsBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 84.6 | |
| DegMat NLI Score entail.Backbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 84.5 | |
| EVL NLI Score entail.Backbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 84.4 | |
| LUQBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 84.4 | |
| FocusBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 84.2 | |
| SATRMDBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 83.6 | |
| DegMat NLI Score entail.Base Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 83.4 | |
| Mean Token EntropyBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 83.2 | |
| Lexical Similarity Rouge-LBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 83 | |
| Semantic DensityBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 82.9 | |
| Semantic DensityBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 82.8 | |
| EVL NLI Score entail.Base Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 82.8 | |
| LookBackLensBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 82.7 | |
| SheepsBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 82.7 | |
| Simple FocusBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 82.6 | |
| SARBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 82.5 | |
| CCPBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 82.4 | |
| LUQBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 82.3 | |
| PerplexityBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 82.2 | |
| CCPBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 82.1 | |
| SARBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 82.1 | |
| Simple FocusBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 82.1 | |
| Semantic EntropyBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 82 | |
| MSPBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 81.9 | |
| DegMat NLI Score entail.Backbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 81.9 | |
| SATRMDBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 81.9 | |
| Ecc. NLI Score entail.Backbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 81.8 | |
| LookBackLensBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 81.6 | |
| Mean Token EntropyBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 81.5 | |
| EVL NLI Score entail.Backbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 81.4 | |
| LUQBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 81.4 | |
| MINDBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 81.2 | |
| MSPBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 80.9 | |
| Ecc. NLI Score entail.Base Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 80.9 | |
| PerplexityBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 80.4 | |
| FocusBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 79.9 | |
| Lexical Similarity Rouge-LBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 79.9 | |
| Ecc. NLI Score entail.Backbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 79.7 | |
| EigenScoreBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 79.7 | |
| Lexical Similarity Rouge-LBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 79.6 | |
| Semantic EntropyBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 79.5 | |
| FocusBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 79.3 | |
| Semantic EntropyBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 79.2 | |
| MINDBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 78.5 | |
| SheepsBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 78.2 | |
| EigenScoreBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 78 | |
| LookBackLensBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 77.8 | |
| SAPLMABackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 77.6 | |
| EigenScoreBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 77.5 | |
| MINDBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 75.6 | |
| SAPLMABackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 74.9 | |
| SAPLMABase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 72.8 | |
| FactoscopeBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=AlignScore2024.08 | 71.6 | |
| FactoscopeBase Model=Llama-3.1 8b, Ground Truth Metric=AlignScore2024.08 | 70.5 | |
| FactoscopeBackbone=Gemma-2 9b, Reference Metric=AlignScore2024.08 | 64 |