List your company

Hallucination Detection on TriviaQA n=1000 (test)

0.794AUROCphi_first
0.60680.65540.7040.7526May 6, 2026

Evaluation results

MethodLinks
phi_firstModel=Llama-3.1-8B, Inference Cost=1 decode2026.050.7940.01664
Sem. AUModel=Llama-3.1-8B, Inference Cost=1+10 decodes (sampling)2026.050.778—64
VerbalizedModel=Qwen2.5-7B, Inference Cost=1 decode2026.050.774—52
phi_firstModel=Qwen2.5-7B, Inference Cost=1 decode2026.050.772-0.00252
AU-3wModel=Llama-3.1-8B, Inference Cost=1+10 decodes (sampling)2026.050.77—64
AU-fullModel=Llama-3.1-8B, Inference Cost=1+10 decodes (sampling)2026.050.769—64
phi_firstModel=TriviaQA mean, Inference Cost=1 decode2026.050.7640.01659
AU-fullModel=Qwen2.5-7B, Inference Cost=1+10 decodes (sampling)2026.050.758—52
AU-1wModel=Llama-3.1-8B, Inference Cost=1+10 decodes (sampling)2026.050.752—64
AU-3wModel=Qwen2.5-7B, Inference Cost=1+10 decodes (sampling)2026.050.751—52
AU-fullModel=TriviaQA mean, Inference Cost=1+10 decodes (sampling)2026.050.748—59
Sem. AUModel=TriviaQA mean, Inference Cost=1+10 decodes (sampling)2026.050.748—59
Sem. AUModel=Qwen2.5-7B, Inference Cost=1+10 decodes (sampling)2026.050.741—52
AU-3wModel=TriviaQA mean, Inference Cost=1+10 decodes (sampling)2026.050.741—59
phi_firstModel=Mistral-7B-v0.3, Inference Cost=1 decode2026.050.7270.00362
AU-1wModel=Qwen2.5-7B, Inference Cost=1+10 decodes (sampling)2026.050.725—52
Sem. AUModel=Mistral-7B-v0.3, Inference Cost=1+10 decodes (sampling)2026.050.724—62
AU-fullModel=Mistral-7B-v0.3, Inference Cost=1+10 decodes (sampling)2026.050.718—62
AU-1wModel=TriviaQA mean, Inference Cost=1+10 decodes (sampling)2026.050.703—59
AU-3wModel=Mistral-7B-v0.3, Inference Cost=1+10 decodes (sampling)2026.050.701—62
VerbalizedModel=Mistral-7B-v0.3, Inference Cost=1 decode2026.050.696—62
VerbalizedModel=TriviaQA mean, Inference Cost=1 decode2026.050.695—59
AU-1wModel=Mistral-7B-v0.3, Inference Cost=1+10 decodes (sampling)2026.050.632—62
VerbalizedModel=Llama-3.1-8B, Inference Cost=1 decode2026.050.614—64