List your company
ResearchFormerly WizWand

Open-ended Generative QA on TruthfulQA (ACC, ECE)

94.6Accuracy (ACC)UQ4CT
83.773686.584389.39592.2057Oct 9, 2024
Updated 6mo ago

Evaluation results

MethodLinks
UQ4CTBackbone=Llama-3.1-8B-Instruct, Fine-tuning=MixLoRA, Number of samples (N)=1, Correctness proxy=F1-based2024.1094.66.08
Verbalized ConsistencyBackbone=Llama-3.1-8B-Instruct, Fine-tuning=MixLoRA, Number of samples (N)=5, Correctness proxy=F1-based2024.1087.4911.35
Semantic EntropyBackbone=Llama-3.1-8B-Instruct, Fine-tuning=MixLoRA, Number of samples (N)=5, Correctness proxy=F1-based2024.1086.6618.19
Semantic DensityBackbone=Llama-3.1-8B-Instruct, Fine-tuning=MixLoRA, Number of samples (N)=5, Correctness proxy=F1-based2024.1086.6616.65
VerbalizedBackbone=Llama-3.1-8B-Instruct, Fine-tuning=MixLoRA, Number of samples (N)=1, Correctness proxy=F1-based2024.1085.6313
SeqProbBackbone=Llama-3.1-8B-Instruct, Fine-tuning=MixLoRA, Number of samples (N)=1, Correctness proxy=F1-based2024.1084.199.93