Hallucination Detection on TriviaQA n=1000 (test)
0.794AUROCphi_first
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| phi_firstModel=Llama-3.1-8B, Inference Cost=1 decode2026.05 | 0.794 | 0.016 | 64 | |
| Sem. AUModel=Llama-3.1-8B, Inference Cost=1+10 decodes (sampling)2026.05 | 0.778 | — | 64 | |
| VerbalizedModel=Qwen2.5-7B, Inference Cost=1 decode2026.05 | 0.774 | — | 52 | |
| phi_firstModel=Qwen2.5-7B, Inference Cost=1 decode2026.05 | 0.772 | -0.002 | 52 | |
| AU-3wModel=Llama-3.1-8B, Inference Cost=1+10 decodes (sampling)2026.05 | 0.77 | — | 64 | |
| AU-fullModel=Llama-3.1-8B, Inference Cost=1+10 decodes (sampling)2026.05 | 0.769 | — | 64 | |
| phi_firstModel=TriviaQA mean, Inference Cost=1 decode2026.05 | 0.764 | 0.016 | 59 | |
| AU-fullModel=Qwen2.5-7B, Inference Cost=1+10 decodes (sampling)2026.05 | 0.758 | — | 52 | |
| AU-1wModel=Llama-3.1-8B, Inference Cost=1+10 decodes (sampling)2026.05 | 0.752 | — | 64 | |
| AU-3wModel=Qwen2.5-7B, Inference Cost=1+10 decodes (sampling)2026.05 | 0.751 | — | 52 | |
| AU-fullModel=TriviaQA mean, Inference Cost=1+10 decodes (sampling)2026.05 | 0.748 | — | 59 | |
| Sem. AUModel=TriviaQA mean, Inference Cost=1+10 decodes (sampling)2026.05 | 0.748 | — | 59 | |
| Sem. AUModel=Qwen2.5-7B, Inference Cost=1+10 decodes (sampling)2026.05 | 0.741 | — | 52 | |
| AU-3wModel=TriviaQA mean, Inference Cost=1+10 decodes (sampling)2026.05 | 0.741 | — | 59 | |
| phi_firstModel=Mistral-7B-v0.3, Inference Cost=1 decode2026.05 | 0.727 | 0.003 | 62 | |
| AU-1wModel=Qwen2.5-7B, Inference Cost=1+10 decodes (sampling)2026.05 | 0.725 | — | 52 | |
| Sem. AUModel=Mistral-7B-v0.3, Inference Cost=1+10 decodes (sampling)2026.05 | 0.724 | — | 62 | |
| AU-fullModel=Mistral-7B-v0.3, Inference Cost=1+10 decodes (sampling)2026.05 | 0.718 | — | 62 | |
| AU-1wModel=TriviaQA mean, Inference Cost=1+10 decodes (sampling)2026.05 | 0.703 | — | 59 | |
| AU-3wModel=Mistral-7B-v0.3, Inference Cost=1+10 decodes (sampling)2026.05 | 0.701 | — | 62 | |
| VerbalizedModel=Mistral-7B-v0.3, Inference Cost=1 decode2026.05 | 0.696 | — | 62 | |
| VerbalizedModel=TriviaQA mean, Inference Cost=1 decode2026.05 | 0.695 | — | 59 | |
| AU-1wModel=Mistral-7B-v0.3, Inference Cost=1+10 decodes (sampling)2026.05 | 0.632 | — | 62 | |
| VerbalizedModel=Llama-3.1-8B, Inference Cost=1 decode2026.05 | 0.614 | — | 64 |