List your company

Hallucination Detection on PopQA n=1000 (test)

0.895AUROCphi_first
0.621480.692490.76350.83451May 6, 2026

Evaluation results

MethodLinks
phi_firstModel=Qwen2.5-7B, Inference Cost=1 decode2026.050.8950.02819
phi_firstModel=Llama-3.1-8B, Inference Cost=1 decode2026.050.8870.01327
phi_firstModel=PopQA mean, Inference Cost=1 decode2026.050.8750.03624
Sem. AUModel=Llama-3.1-8B, Inference Cost=1+10 decodes (sampling)2026.050.874—27
Sem. AUModel=Qwen2.5-7B, Inference Cost=1+10 decodes (sampling)2026.050.867—19
AU-fullModel=Llama-3.1-8B, Inference Cost=1+10 decodes (sampling)2026.050.866—27
AU-fullModel=Qwen2.5-7B, Inference Cost=1+10 decodes (sampling)2026.050.857—19
AU-3wModel=Qwen2.5-7B, Inference Cost=1+10 decodes (sampling)2026.050.856—19
AU-3wModel=Llama-3.1-8B, Inference Cost=1+10 decodes (sampling)2026.050.854—27
phi_firstModel=Mistral-7B-v0.3, Inference Cost=1 decode2026.050.8420.06425
AU-1wModel=Llama-3.1-8B, Inference Cost=1+10 decodes (sampling)2026.050.84—27
Sem. AUModel=PopQA mean, Inference Cost=1+10 decodes (sampling)2026.050.839—24
AU-fullModel=PopQA mean, Inference Cost=1+10 decodes (sampling)2026.050.834—24
AU-1wModel=Qwen2.5-7B, Inference Cost=1+10 decodes (sampling)2026.050.831—19
AU-3wModel=PopQA mean, Inference Cost=1+10 decodes (sampling)2026.050.824—24
AU-1wModel=PopQA mean, Inference Cost=1+10 decodes (sampling)2026.050.802—24
VerbalizedModel=Qwen2.5-7B, Inference Cost=1 decode2026.050.782—19
AU-fullModel=Mistral-7B-v0.3, Inference Cost=1+10 decodes (sampling)2026.050.778—25
Sem. AUModel=Mistral-7B-v0.3, Inference Cost=1+10 decodes (sampling)2026.050.775—25
AU-3wModel=Mistral-7B-v0.3, Inference Cost=1+10 decodes (sampling)2026.050.762—25
AU-1wModel=Mistral-7B-v0.3, Inference Cost=1+10 decodes (sampling)2026.050.735—25
VerbalizedModel=PopQA mean, Inference Cost=1 decode2026.050.705—24
VerbalizedModel=Mistral-7B-v0.3, Inference Cost=1 decode2026.050.701—25
VerbalizedModel=Llama-3.1-8B, Inference Cost=1 decode2026.050.632—27