Hallucination Detection on PopQA n=1000 (test)
0.895AUROCphi_first
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| phi_firstModel=Qwen2.5-7B, Inference Cost=1 decode2026.05 | 0.895 | 0.028 | 19 | |
| phi_firstModel=Llama-3.1-8B, Inference Cost=1 decode2026.05 | 0.887 | 0.013 | 27 | |
| phi_firstModel=PopQA mean, Inference Cost=1 decode2026.05 | 0.875 | 0.036 | 24 | |
| Sem. AUModel=Llama-3.1-8B, Inference Cost=1+10 decodes (sampling)2026.05 | 0.874 | — | 27 | |
| Sem. AUModel=Qwen2.5-7B, Inference Cost=1+10 decodes (sampling)2026.05 | 0.867 | — | 19 | |
| AU-fullModel=Llama-3.1-8B, Inference Cost=1+10 decodes (sampling)2026.05 | 0.866 | — | 27 | |
| AU-fullModel=Qwen2.5-7B, Inference Cost=1+10 decodes (sampling)2026.05 | 0.857 | — | 19 | |
| AU-3wModel=Qwen2.5-7B, Inference Cost=1+10 decodes (sampling)2026.05 | 0.856 | — | 19 | |
| AU-3wModel=Llama-3.1-8B, Inference Cost=1+10 decodes (sampling)2026.05 | 0.854 | — | 27 | |
| phi_firstModel=Mistral-7B-v0.3, Inference Cost=1 decode2026.05 | 0.842 | 0.064 | 25 | |
| AU-1wModel=Llama-3.1-8B, Inference Cost=1+10 decodes (sampling)2026.05 | 0.84 | — | 27 | |
| Sem. AUModel=PopQA mean, Inference Cost=1+10 decodes (sampling)2026.05 | 0.839 | — | 24 | |
| AU-fullModel=PopQA mean, Inference Cost=1+10 decodes (sampling)2026.05 | 0.834 | — | 24 | |
| AU-1wModel=Qwen2.5-7B, Inference Cost=1+10 decodes (sampling)2026.05 | 0.831 | — | 19 | |
| AU-3wModel=PopQA mean, Inference Cost=1+10 decodes (sampling)2026.05 | 0.824 | — | 24 | |
| AU-1wModel=PopQA mean, Inference Cost=1+10 decodes (sampling)2026.05 | 0.802 | — | 24 | |
| VerbalizedModel=Qwen2.5-7B, Inference Cost=1 decode2026.05 | 0.782 | — | 19 | |
| AU-fullModel=Mistral-7B-v0.3, Inference Cost=1+10 decodes (sampling)2026.05 | 0.778 | — | 25 | |
| Sem. AUModel=Mistral-7B-v0.3, Inference Cost=1+10 decodes (sampling)2026.05 | 0.775 | — | 25 | |
| AU-3wModel=Mistral-7B-v0.3, Inference Cost=1+10 decodes (sampling)2026.05 | 0.762 | — | 25 | |
| AU-1wModel=Mistral-7B-v0.3, Inference Cost=1+10 decodes (sampling)2026.05 | 0.735 | — | 25 | |
| VerbalizedModel=PopQA mean, Inference Cost=1 decode2026.05 | 0.705 | — | 24 | |
| VerbalizedModel=Mistral-7B-v0.3, Inference Cost=1 decode2026.05 | 0.701 | — | 25 | |
| VerbalizedModel=Llama-3.1-8B, Inference Cost=1 decode2026.05 | 0.632 | — | 27 |