Hallucination Detection on MetaQA 1-hop LLaMA2-7B responses (val)
83.41AUCSSR + SAS
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| SSR + SASCategory=Mechanistic Detector2026.05 | 83.41 | 53.9 | 75.24 | 56.32 | 51.68 | |
| Embedding DivergenceCategory=Semantic Consistency2026.05 | 69.14 | 29.55 | 50.94 | 19.14 | 64.76 | |
| BERTScoreCategory=Semantic Consistency2026.05 | 63.66 | 27.4 | 50.92 | 18.2 | 55.44 | |
| NLI ContradictionCategory=Semantic Consistency2026.05 | 57.41 | 25.67 | 35.24 | 15.12 | 36.67 | |
| Token confidenceCategory=Model-Internal Confidence2026.05 | 50.2 | 20.2 | 45.29 | 13.24 | 42.49 | |
| Max token probabilityCategory=Model-Internal Confidence2026.05 | 50 | 29.6 | 44.78 | 13.22 | 44.04 | |
| PerplexityCategory=Model-Internal Confidence2026.05 | 45.78 | 25.6 | 21.26 | 14.84 | 93.1 |