Selective Generation on WMT19
0.845ROC-AUCLookBackLens
Evaluation Results
| Method | Links | |
|---|---|---|
| LookBackLensBackbone=Gemma-2 9b, Reference Metric=Comet2024.08 | 0.845 | |
| SAPLMABackbone=Gemma-2 9b, Reference Metric=Comet2024.08 | 0.831 | |
| SheepsBackbone=Gemma-2 9b, Reference Metric=Comet2024.08 | 0.831 | |
| MINDBackbone=Gemma-2 9b, Reference Metric=Comet2024.08 | 0.826 | |
| TADBackbone=Gemma-2 9b, Reference Metric=Comet2024.08 | 0.824 | |
| LookBackLensBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.08 | 0.82 | |
| TADBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.08 | 0.812 | |
| TADBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.08 | 0.795 | |
| SAPLMABase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.08 | 0.792 | |
| SheepsBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.08 | 0.778 | |
| SATRMDBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.08 | 0.774 | |
| SATRMDBackbone=Gemma-2 9b, Reference Metric=Comet2024.08 | 0.774 | |
| LookBackLensBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.08 | 0.772 | |
| Simple FocusBackbone=Gemma-2 9b, Reference Metric=Comet2024.08 | 0.761 | |
| Mean Token EntropyBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.08 | 0.76 | |
| SheepsBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.08 | 0.76 | |
| PerplexityBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.08 | 0.755 | |
| Simple FocusBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.08 | 0.751 | |
| MINDBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.08 | 0.748 | |
| SARBackbone=Gemma-2 9b, Reference Metric=Comet2024.08 | 0.745 | |
| SARBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.08 | 0.743 | |
| Simple FocusBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.08 | 0.738 | |
| Mean Token EntropyBackbone=Gemma-2 9b, Reference Metric=Comet2024.08 | 0.734 | |
| MSPBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.08 | 0.733 | |
| PerplexityBackbone=Gemma-2 9b, Reference Metric=Comet2024.08 | 0.733 | |
| MSPBackbone=Gemma-2 9b, Reference Metric=Comet2024.08 | 0.732 | |
| SATRMDBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.08 | 0.729 | |
| Mean Token EntropyBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.08 | 0.728 | |
| SARBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.08 | 0.727 | |
| MSPBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.08 | 0.726 | |
| FocusBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.08 | 0.725 | |
| PerplexityBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.08 | 0.722 | |
| FocusBackbone=Gemma-2 9b, Reference Metric=Comet2024.08 | 0.721 | |
| Lexical Similarity Rouge-LBackbone=Gemma-2 9b, Reference Metric=Comet2024.08 | 0.713 | |
| Lexical Similarity Rouge-LBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.08 | 0.709 | |
| FocusBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.08 | 0.708 | |
| Semantic EntropyBackbone=Gemma-2 9b, Reference Metric=Comet2024.08 | 0.706 | |
| Semantic EntropyBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.08 | 0.703 | |
| Lexical Similarity Rouge-LBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.08 | 0.703 | |
| EigenScoreBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.08 | 0.702 | |
| Semantic EntropyBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.08 | 0.692 | |
| CCPBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.08 | 0.688 | |
| Ecc. NLI Score entail.Backbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.08 | 0.688 | |
| MINDBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.08 | 0.682 | |
| Ecc. NLI Score entail.Backbone=Gemma-2 9b, Reference Metric=Comet2024.08 | 0.681 | |
| CCPBackbone=Gemma-2 9b, Reference Metric=Comet2024.08 | 0.679 | |
| Semantic DensityBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.08 | 0.677 | |
| DegMat NLI Score entail.Backbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.08 | 0.675 | |
| DegMat NLI Score entail.Backbone=Gemma-2 9b, Reference Metric=Comet2024.08 | 0.671 | |
| CCPBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.08 | 0.671 | |
| EVL NLI Score entail.Backbone=Gemma-2 9b, Reference Metric=Comet2024.08 | 0.668 | |
| EigenScoreBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.08 | 0.668 | |
| SAPLMABackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.08 | 0.667 | |
| EVL NLI Score entail.Backbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.08 | 0.665 | |
| Semantic DensityBackbone=Gemma-2 9b, Reference Metric=Comet2024.08 | 0.661 | |
| Ecc. NLI Score entail.Base Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.08 | 0.654 | |
| LUQBackbone=Gemma-2 9b, Reference Metric=Comet2024.08 | 0.648 | |
| LUQBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.08 | 0.647 | |
| DegMat NLI Score entail.Base Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.08 | 0.64 | |
| EigenScoreBackbone=Gemma-2 9b, Reference Metric=Comet2024.08 | 0.639 | |
| Semantic DensityBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.08 | 0.638 | |
| EVL NLI Score entail.Base Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.08 | 0.63 | |
| LUQBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.08 | 0.629 | |
| FactoscopeBase Model=Llama-3.1 8b, Ground Truth Metric=Comet2024.08 | 0.592 | |
| FactoscopeBackbone=Qwen-2.5 7b, UQ Level=Sequence-level, Reference Metric=Comet2024.08 | 0.54 | |
| FactoscopeBackbone=Gemma-2 9b, Reference Metric=Comet2024.08 | 0.529 |