Uncertainty Estimation on TriviaQA, SVAMP, and NQ Average
0.843AUROCG-NLL
Evaluation Results
| Method | Links | |
|---|---|---|
| G-NLLArchitecture=State-Space, Model Size=7B, Training Stage=PT, Generation Length=short, Correctness Metric=F1, Scoring Rule=Zero-One2024.12 | 0.843 | |
| G-NLLArchitecture=Transformer, Model Size=8B, Training Stage=IT, Generation Length=short, Correctness Metric=F1, Scoring Rule=Zero-One2024.12 | 0.838 | |
| G-NLLArchitecture=State-Space, Model Size=7B, Training Stage=IT, Generation Length=short, Correctness Metric=F1, Scoring Rule=Zero-One2024.12 | 0.838 | |
| G-NLLArchitecture=Transformer, Model Size=8B, Training Stage=PT, Generation Length=short, Correctness Metric=F1, Scoring Rule=Zero-One2024.12 | 0.824 | |
| G-NLLArchitecture=Transformer, Model Size=70B, Training Stage=PT, Generation Length=short, Correctness Metric=F1, Scoring Rule=Zero-One2024.12 | 0.82 | |
| D-SEArchitecture=Transformer, Model Size=8B, Training Stage=PT, Generation Length=short, Correctness Metric=F1, Scoring Rule=Logarithmic2024.12 | 0.804 | |
| LN-PEArchitecture=Transformer, Model Size=8B, Training Stage=PT, Generation Length=short, Correctness Metric=F1, Scoring Rule=Logarithmic2024.12 | 0.795 | |
| LN-SEArchitecture=Transformer, Model Size=8B, Training Stage=PT, Generation Length=short, Correctness Metric=F1, Scoring Rule=Logarithmic2024.12 | 0.793 | |
| G-NLLArchitecture=Transformer, Model Size=70B, Training Stage=IT, Generation Length=short, Correctness Metric=F1, Scoring Rule=Zero-One2024.12 | 0.792 | |
| PEArchitecture=Transformer, Model Size=8B, Training Stage=PT, Generation Length=short, Correctness Metric=F1, Scoring Rule=Logarithmic2024.12 | 0.776 | |
| SEArchitecture=Transformer, Model Size=8B, Training Stage=PT, Generation Length=short, Correctness Metric=F1, Scoring Rule=Logarithmic2024.12 | 0.775 | |
| G-NLLArchitecture=State-Space, Model Size=7B, Training Stage=PT, Generation Length=short, Correctness Metric=LLM, Scoring Rule=Zero-One2024.12 | 0.728 | |
| G-NLLArchitecture=Transformer, Model Size=8B, Training Stage=PT, Generation Length=short, Correctness Metric=LLM, Scoring Rule=Zero-One2024.12 | 0.726 | |
| G-NLLArchitecture=Transformer, Model Size=70B, Training Stage=PT, Generation Length=short, Correctness Metric=LLM, Scoring Rule=Zero-One2024.12 | 0.723 | |
| G-NLLArchitecture=Transformer, Model Size=8B, Training Stage=IT, Generation Length=short, Correctness Metric=LLM, Scoring Rule=Zero-One2024.12 | 0.722 | |
| G-NLLArchitecture=State-Space, Model Size=7B, Training Stage=IT, Generation Length=short, Correctness Metric=LLM, Scoring Rule=Zero-One2024.12 | 0.719 | |
| G-NLLArchitecture=Transformer, Model Size=70B, Training Stage=IT, Generation Length=short, Correctness Metric=LLM, Scoring Rule=Zero-One2024.12 | 0.699 | |
| G-NLLArchitecture=Transformer, Model Size=8B, Training Stage=PT, Generation Length=long, Correctness Metric=LLM, Scoring Rule=Zero-One2024.12 | 0.649 | |
| G-NLLArchitecture=Transformer, Model Size=70B, Training Stage=PT, Generation Length=long, Correctness Metric=LLM, Scoring Rule=Zero-One2024.12 | 0.649 | |
| G-NLLArchitecture=Transformer, Model Size=8B, Training Stage=IT, Generation Length=long, Correctness Metric=LLM, Scoring Rule=Zero-One2024.12 | 0.615 | |
| G-NLLArchitecture=State-Space, Model Size=7B, Training Stage=IT, Generation Length=long, Correctness Metric=LLM, Scoring Rule=Zero-One2024.12 | 0.615 | |
| G-NLLArchitecture=State-Space, Model Size=7B, Training Stage=PT, Generation Length=long, Correctness Metric=LLM, Scoring Rule=Zero-One2024.12 | 0.612 | |
| G-NLLArchitecture=Transformer, Model Size=70B, Training Stage=IT, Generation Length=long, Correctness Metric=LLM, Scoring Rule=Zero-One2024.12 | 0.562 |