List your company
ResearchFormerly WizWand

Uncertainty Estimation on TriviaQA (test)

87.91AUROCACT-ViT
55.09863.616572.13580.6535Jul 3, 2023Dec 26, 2023Jun 20, 2024Dec 14, 2024Jun 8, 2025Dec 2, 2025May 28, 2026
Updated 5mo ago

Evaluation results

MethodLinks
ACT-ViTTrain Dataset=HotpotQA, Model=Mistral-7B-Instruct-v0.32026.0387.91———
CAGE-CALRollouts=3, Topologies=52026.0586.12——93.02
TSSC Measure=IC-SC, Model=Qwen, Evaluation Protocol=SE_vanilla (greedy-decoding)2026.0485.7———
TSSC Measure=E-SC, Model=Qwen, Evaluation Protocol=SE_vanilla (greedy-decoding)2026.0484.9———
TSSC Measure=G-SC, Model=Qwen, Evaluation Protocol=SE_vanilla (greedy-decoding)2026.0484.8———
TSSC Measure=L-SC, Model=Qwen, Evaluation Protocol=SE_vanilla (greedy-decoding)2026.0484.4———
ATSSC Measure=E-SC, Model=Qwen, Evaluation Protocol=SE_vanilla (greedy-decoding)2026.0483.4———
ATSSC Measure=G-SC, Model=Qwen, Evaluation Protocol=SE_vanilla (greedy-decoding)2026.0483.4———
SESC Measure=E-SC, Model=Qwen, Evaluation Protocol=SE_vanilla (greedy-decoding)2026.0483.3———
SESC Measure=L-SC, Model=Qwen, Evaluation Protocol=SE_vanilla (greedy-decoding)2026.0483.3———
SESC Measure=IC-SC, Model=Qwen, Evaluation Protocol=SE_vanilla (greedy-decoding)2026.0483.3———
SESC Measure=G-SC, Model=Qwen, Evaluation Protocol=SE_vanilla (greedy-decoding)2026.0483.3———
PlattSC Measure=E-SC, Model=Qwen, Evaluation Protocol=SE_vanilla (greedy-decoding)2026.0483.2———
PlattSC Measure=L-SC, Model=Qwen, Evaluation Protocol=SE_vanilla (greedy-decoding)2026.0483.2———
PlattSC Measure=IC-SC, Model=Qwen, Evaluation Protocol=SE_vanilla (greedy-decoding)2026.0483.2———
PlattSC Measure=G-SC, Model=Qwen, Evaluation Protocol=SE_vanilla (greedy-decoding)2026.0483.2———
ATSSC Measure=L-SC, Model=Qwen, Evaluation Protocol=SE_vanilla (greedy-decoding)2026.0483.2———
ATSSC Measure=IC-SC, Model=Qwen, Evaluation Protocol=SE_vanilla (greedy-decoding)2026.0483.2———
ACT-ViTTrain Dataset=TriviaQA, Model=Mistral-7B-Instruct-v0.32026.0382.66———
SETarget Model=LLaMA-3-8B-Instruct, Setting=+Corrector2025.0582.1212.761.2—
SARTarget Model=LLaMA-3-8B-Instruct, Setting=+Corrector2025.0581.913.760.98—
Plurality voteRollouts=3, Topologies=52026.0581.89——93.09
SARModel=LLaMA-7b, Rouge-L threshold=0.32023.0781.8———
SEModel=LLaMA-7b, Rouge-L threshold=0.32023.0781.4———
SETarget Model=OPT-6.7B, Setting=+Corrector2025.058127.050.34—
SETarget Model=LLaMA-3-8B-Instruct, Setting=Vanilla2025.0580.9213.07——
SARTarget Model=LLaMA-3-8B-Instruct, Setting=Vanilla2025.0580.9216.17——
SETarget Model=OPT-6.7B, Setting=Vanilla2025.0580.6636.64——
SAR-tTarget Model=OPT-6.7B, Setting=+Corrector2025.0580.2137.851.97—
Answer entropyRollouts=3, Topologies=52026.0580.19——92.82
DAERollouts=3, Topologies=52026.0580.19——92.82
SENTSARModel=LLaMA-7b, Rouge-L threshold=0.32023.0780———
LN-PETarget Model=OPT-6.7B, Setting=+Corrector2025.0579.9320.81.57—
SAR-tTarget Model=LLaMA-3-8B-Instruct, Setting=+Corrector2025.0579.9313.70.38—
SAR-tTarget Model=LLaMA-3-8B-Instruct, Setting=Vanilla2025.0579.5516.4——
TOKENSARModel=LLaMA-7b, Rouge-L threshold=0.32023.0779.3———
SARTarget Model=OPT-6.7B, Setting=+Corrector2025.0578.6731.023.35—
LN-PETarget Model=OPT-6.7B, Setting=Vanilla2025.0578.3732.29——
LN-PEModel=LLaMA-7b, Rouge-L threshold=0.32023.0778.3———
SAR-tTarget Model=OPT-6.7B, Setting=Vanilla2025.0578.2440.14——
SAR-sTarget Model=LLaMA-3-8B-Instruct, Setting=+Corrector2025.0577.09202.95—
SEModel=LLaMA-13b, Rouge-L threshold=0.32023.0775.8———
BaseSC Measure=E-SC, Model=Qwen, Evaluation Protocol=SE_vanilla (greedy-decoding)2026.0475.5———
BaseSC Measure=L-SC, Model=Qwen, Evaluation Protocol=SE_vanilla (greedy-decoding)2026.0475.5———
BaseSC Measure=G-SC, Model=Qwen, Evaluation Protocol=SE_vanilla (greedy-decoding)2026.0475.5———
BaseSC Measure=IC-SC, Model=Qwen, Evaluation Protocol=SE_vanilla (greedy-decoding)2026.0475.4———
SARTarget Model=OPT-6.7B, Setting=Vanilla2025.0575.3240.61——
SENTSARModel=LLaMA-13b, Rouge-L threshold=0.32023.0774.9———
SARLLM Model=Vicuna-13b, Number of generations=5, Rouge-L threshold=0.52023.0774.9———
VCTarget Model=LLaMA-3-8B-Instruct, Setting=+Corrector2025.0574.8916.7812.55—
LN-PETarget Model=LLaMA-3-8B-Instruct, Setting=+Corrector2025.0574.7911.532.24—
SENTSARLLM Model=Vicuna-13b, Number of generations=5, Rouge-L threshold=0.52023.0774.5———
SARLLM Model=WizardLM-13b, Number of generations=5, Rouge-L threshold=0.52023.0774.4———
SENTSARLLM Model=WizardLM-13b, Number of generations=5, Rouge-L threshold=0.52023.0774.3———
LN-PETarget Model=LLaMA-3-8B-Instruct, Setting=Vanilla2025.0572.5514.31——
P(True)Target Model=OPT-6.7B, Setting=+Corrector2025.0572.2932.635.84—
P(True)Target Model=LLaMA-3-8B-Instruct, Setting=+Corrector2025.0572.2919.8415.15—
SARModel=LLaMA-13b, Rouge-L threshold=0.32023.0771.6———
SENTSARLLM Model=Vicuna-33b, Number of generations=5, Rouge-L threshold=0.52023.0771.5———
PEModel=LLaMA-7b, Rouge-L threshold=0.32023.0771.3———
MATURollouts=3, Topologies=52026.0571.23——89.73
SignaturesTrain Dataset=IMDB, Model=Mistral-7B-Instruct-v0.32026.0371.19———
SARLLM Model=Vicuna-33b, Number of generations=5, Rouge-L threshold=0.52023.0771———
VCTarget Model=OPT-6.7B, Setting=+Corrector2025.0570.5527.6110.15—
SARLLM Model=LLaMA-2-13b-chat, Number of generations=5, Rouge-L threshold=0.52023.0770.4———
SAR-sTarget Model=LLaMA-3-8B-Instruct, Setting=Vanilla2025.0569.8723.17——
Avg-logprobRollouts=3, Topologies=52026.0569.83——86.96
LSTarget Model=LLaMA-3-8B-Instruct, Setting=+Corrector2025.0569.827.4150.25—
SENTSARLLM Model=LLaMA-2-13b-chat, Number of generations=5, Rouge-L threshold=0.52023.0769.8———
PETarget Model=LLaMA-3-8B-Instruct, Setting=+Corrector2025.0569.7617.245.25—
TOKENSARLLM Model=Vicuna-13b, Number of generations=5, Rouge-L threshold=0.52023.0769.2———
PELLM Model=Vicuna-13b, Number of generations=5, Rouge-L threshold=0.52023.0769———
P(True)Target Model=OPT-6.7B, Setting=Vanilla2025.0566.7445——
PETarget Model=OPT-6.7B, Setting=+Corrector2025.0566.6220.2810.25—
TOKENSARLLM Model=WizardLM-13b, Number of generations=5, Rouge-L threshold=0.52023.0765.7———
LSModel=LLaMA-13b, Rouge-L threshold=0.32023.0765.5———
TOKENSARLLM Model=LLaMA-2-13b-chat, Number of generations=5, Rouge-L threshold=0.52023.0765.4———
TOKENSARLLM Model=Vicuna-33b, Number of generations=5, Rouge-L threshold=0.52023.0765.2———
LSTarget Model=OPT-6.7B, Setting=+Corrector2025.0565.1141.7618.62—
SELLM Model=Vicuna-33b, Number of generations=5, Rouge-L threshold=0.52023.0765.1———
PELLM Model=WizardLM-13b, Number of generations=5, Rouge-L threshold=0.52023.0764.7———
PELLM Model=LLaMA-2-13b-chat, Number of generations=5, Rouge-L threshold=0.52023.0764.7———
PETarget Model=LLaMA-3-8B-Instruct, Setting=Vanilla2025.0564.5221.38——
PELLM Model=Vicuna-33b, Number of generations=5, Rouge-L threshold=0.52023.0764.4———
LN-PELLM Model=Vicuna-33b, Number of generations=5, Rouge-L threshold=0.52023.0763.9———
TOKENSARModel=LLaMA-13b, Rouge-L threshold=0.32023.0763.5———
SELLM Model=WizardLM-13b, Number of generations=5, Rouge-L threshold=0.52023.0763.4———
SELLM Model=Vicuna-13b, Number of generations=5, Rouge-L threshold=0.52023.0763———
LN-PEModel=LLaMA-13b, Rouge-L threshold=0.32023.0762.7———
LN-PELLM Model=Vicuna-13b, Number of generations=5, Rouge-L threshold=0.52023.0762.4———
VCTarget Model=LLaMA-3-8B-Instruct, Setting=Vanilla2025.0562.3423.41——
SELLM Model=LLaMA-2-13b-chat, Number of generations=5, Rouge-L threshold=0.52023.0762.2———
ACT-ViTTrain Dataset=IMDB, Model=Mistral-7B-Instruct-v0.32026.0362.01———
LN-PELLM Model=WizardLM-13b, Number of generations=5, Rouge-L threshold=0.52023.0761.5———
LN-PELLM Model=LLaMA-2-13b-chat, Number of generations=5, Rouge-L threshold=0.52023.0761.5———
VCTarget Model=OPT-6.7B, Setting=Vanilla2025.0560.4149.13——
SignaturesTrain Dataset=HotpotQA, Model=Mistral-7B-Instruct-v0.32026.0358.42———
P(True)Target Model=LLaMA-3-8B-Instruct, Setting=Vanilla2025.0557.1424.67——
LSLLM Model=Vicuna-33b, Number of generations=5, Rouge-L threshold=0.52023.0756.5———
PETarget Model=OPT-6.7B, Setting=Vanilla2025.0556.3642.39——