List your company
ResearchFormerly WizWand

Uncertainty Estimation on TriviaQA, SVAMP, and NQ Average

0.843AUROCG-NLL
0.550760.626630.70250.77837Dec 19, 2024
Updated 6mo ago

Evaluation results

MethodLinks
G-NLLArchitecture=State-Space, Model Size=7B, Training Stage=PT, Generation Length=short, Correctness Metric=F1, Scoring Rule=Zero-One2024.120.843
G-NLLArchitecture=Transformer, Model Size=8B, Training Stage=IT, Generation Length=short, Correctness Metric=F1, Scoring Rule=Zero-One2024.120.838
G-NLLArchitecture=State-Space, Model Size=7B, Training Stage=IT, Generation Length=short, Correctness Metric=F1, Scoring Rule=Zero-One2024.120.838
G-NLLArchitecture=Transformer, Model Size=8B, Training Stage=PT, Generation Length=short, Correctness Metric=F1, Scoring Rule=Zero-One2024.120.824
G-NLLArchitecture=Transformer, Model Size=70B, Training Stage=PT, Generation Length=short, Correctness Metric=F1, Scoring Rule=Zero-One2024.120.82
D-SEArchitecture=Transformer, Model Size=8B, Training Stage=PT, Generation Length=short, Correctness Metric=F1, Scoring Rule=Logarithmic2024.120.804
LN-PEArchitecture=Transformer, Model Size=8B, Training Stage=PT, Generation Length=short, Correctness Metric=F1, Scoring Rule=Logarithmic2024.120.795
LN-SEArchitecture=Transformer, Model Size=8B, Training Stage=PT, Generation Length=short, Correctness Metric=F1, Scoring Rule=Logarithmic2024.120.793
G-NLLArchitecture=Transformer, Model Size=70B, Training Stage=IT, Generation Length=short, Correctness Metric=F1, Scoring Rule=Zero-One2024.120.792
PEArchitecture=Transformer, Model Size=8B, Training Stage=PT, Generation Length=short, Correctness Metric=F1, Scoring Rule=Logarithmic2024.120.776
SEArchitecture=Transformer, Model Size=8B, Training Stage=PT, Generation Length=short, Correctness Metric=F1, Scoring Rule=Logarithmic2024.120.775
G-NLLArchitecture=State-Space, Model Size=7B, Training Stage=PT, Generation Length=short, Correctness Metric=LLM, Scoring Rule=Zero-One2024.120.728
G-NLLArchitecture=Transformer, Model Size=8B, Training Stage=PT, Generation Length=short, Correctness Metric=LLM, Scoring Rule=Zero-One2024.120.726
G-NLLArchitecture=Transformer, Model Size=70B, Training Stage=PT, Generation Length=short, Correctness Metric=LLM, Scoring Rule=Zero-One2024.120.723
G-NLLArchitecture=Transformer, Model Size=8B, Training Stage=IT, Generation Length=short, Correctness Metric=LLM, Scoring Rule=Zero-One2024.120.722
G-NLLArchitecture=State-Space, Model Size=7B, Training Stage=IT, Generation Length=short, Correctness Metric=LLM, Scoring Rule=Zero-One2024.120.719
G-NLLArchitecture=Transformer, Model Size=70B, Training Stage=IT, Generation Length=short, Correctness Metric=LLM, Scoring Rule=Zero-One2024.120.699
G-NLLArchitecture=Transformer, Model Size=8B, Training Stage=PT, Generation Length=long, Correctness Metric=LLM, Scoring Rule=Zero-One2024.120.649
G-NLLArchitecture=Transformer, Model Size=70B, Training Stage=PT, Generation Length=long, Correctness Metric=LLM, Scoring Rule=Zero-One2024.120.649
G-NLLArchitecture=Transformer, Model Size=8B, Training Stage=IT, Generation Length=long, Correctness Metric=LLM, Scoring Rule=Zero-One2024.120.615
G-NLLArchitecture=State-Space, Model Size=7B, Training Stage=IT, Generation Length=long, Correctness Metric=LLM, Scoring Rule=Zero-One2024.120.615
G-NLLArchitecture=State-Space, Model Size=7B, Training Stage=PT, Generation Length=long, Correctness Metric=LLM, Scoring Rule=Zero-One2024.120.612
G-NLLArchitecture=Transformer, Model Size=70B, Training Stage=IT, Generation Length=long, Correctness Metric=LLM, Scoring Rule=Zero-One2024.120.562