Hallucination self-detection on SimpleQA
95.9AUROCAdaptive Bayesian Estimation
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Adaptive Bayesian EstimationLLM=Llama-2-7B, Sampling budget (N)=52026.03 | 95.9 | — | — | — | — | |
| SESDLGLLM=Llama-2-7B, Sampling budget (N)=52026.03 | 95.6 | — | — | — | — | |
| Adaptive Bayesian EstimationLLM=Llama-3.1-8B, Sampling budget (N)=52026.03 | 94.2 | — | — | — | — | |
| SESDLGLLM=Llama-3.1-8B, Sampling budget (N)=52026.03 | 93.4 | — | — | — | — | |
| SELLM=Llama-2-7B, Sampling budget (N)=52026.03 | 93 | — | — | — | — | |
| SELLM=Llama-3.1-8B, Sampling budget (N)=52026.03 | 93 | — | — | — | — | |
| Adaptive Bayesian EstimationLLM=Llama-3.1-8B, Sampling budget (N)=22026.03 | 91.3 | — | — | — | — | |
| Adaptive Bayesian EstimationLLM=Llama-2-7B, Sampling budget (N)=22026.03 | 89.5 | — | — | — | — | |
| SESDLGLLM=Llama-2-7B, Sampling budget (N)=22026.03 | 89.1 | — | — | — | — | |
| SESDLGLLM=Mistral-Small-24B, Sampling budget (N)=52026.03 | 87.3 | — | — | — | — | |
| Adaptive Bayesian EstimationLLM=Mistral-Small-24B, Sampling budget (N)=52026.03 | 87.1 | — | — | — | — | |
| SELLM=Mistral-Small-24B, Sampling budget (N)=52026.03 | 86.8 | — | — | — | — | |
| SESDLGLLM=Llama-3.1-8B, Sampling budget (N)=22026.03 | 83 | — | — | — | — | |
| SELLM=Llama-3.1-8B, Sampling budget (N)=22026.03 | 82.5 | — | — | — | — | |
| SELLM=Llama-2-7B, Sampling budget (N)=22026.03 | 79.6 | — | — | — | — | |
| Adaptive Bayesian EstimationLLM=Mistral-Small-24B, Sampling budget (N)=22026.03 | 77.2 | — | — | — | — | |
| SARLLM=Llama-2-7B, Sampling budget (N)=52026.03 | 76.8 | — | — | — | — | |
| SESDLGLLM=Mistral-Small-24B, Sampling budget (N)=22026.03 | 73.7 | — | — | — | — | |
| SELLM=Mistral-Small-24B, Sampling budget (N)=22026.03 | 72.5 | — | — | — | — | |
| SARLLM=Llama-2-7B, Sampling budget (N)=22026.03 | 72.1 | — | — | — | — | |
| SARLLM=Mistral-Small-24B, Sampling budget (N)=52026.03 | 68.1 | — | — | — | — | |
| P(True)LLM=Mistral-Small-24B2026.03 | 66.8 | — | — | — | — | |
| P(True)LLM=Llama-3.1-8B2026.03 | 65.5 | — | — | — | — | |
| SARLLM=Llama-3.1-8B, Sampling budget (N)=22026.03 | 65.2 | — | — | — | — | |
| SARLLM=Mistral-Small-24B, Sampling budget (N)=22026.03 | 64.5 | — | — | — | — | |
| SARLLM=Llama-3.1-8B, Sampling budget (N)=52026.03 | 61.4 | — | — | — | — | |
| P(True)LLM=Llama-2-7B2026.03 | 52.1 | — | — | — | — | |
| GPT-4o2024.12 | — | 38 | — | — | 61 | |
| GPT-4o-mini2024.12 | — | 9 | — | — | 90 | |
| o12024.12 | — | 47 | — | — | 44 | |
| o1-mini2024.12 | — | 7 | — | — | 60 | |
| o1-preview2024.12 | — | 42 | — | — | 44 | |
| Self-ConfidenceModel=GPT-4o, Sampling Strategy=Single-Sample, Knowledge Graph Support=Disabled2025.12 | — | 57 | 57 | 50 | — | |
| Self-ConfidenceModel=GPT-4o, Sampling Strategy=Single-Sample, Knowledge Graph Support=Enabled2025.12 | — | 64 | 63 | 61 | — | |
| Self-ConfidenceModel=Gemini-2.5-Flash, Sampling Strategy=Single-Sample, Knowledge Graph Support=Disabled2025.12 | — | 53 | 57 | 53 | — | |
| Self-ConfidenceModel=Gemini-2.5-Flash, Sampling Strategy=Single-Sample, Knowledge Graph Support=Enabled2025.12 | — | 56 | 59 | 53 | — | |
| Self-QuestioningModel=GPT-4o, Sampling Strategy=Single-Sample, Knowledge Graph Support=Disabled2025.12 | — | 60 | 51 | 57 | — | |
| Self-QuestioningModel=GPT-4o, Sampling Strategy=Single-Sample, Knowledge Graph Support=Enabled2025.12 | — | 67 | 60 | 69 | — | |
| Self-QuestioningModel=Gemini-2.5-Flash, Sampling Strategy=Single-Sample, Knowledge Graph Support=Disabled2025.12 | — | 62 | 49 | 50 | — | |
| Self-QuestioningModel=Gemini-2.5-Flash, Sampling Strategy=Single-Sample, Knowledge Graph Support=Enabled2025.12 | — | 63 | 59 | 62 | — |