# The First Token Knows: Single-Decode Confidence for Hallucination Detection

Source: [The First Token Knows: Single-Decode Confidence for Hallucination Detection](<https://www.sota2.com/research/paper/69fc1cc45497b3b670c88788>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-10T06:00:01.405Z

Authors: Mina Gabriel

Published: 2026-05-06T17:34:00.000Z

## Abstract

Self-consistency detects hallucinations by generating multiple sampled answers to a question and measuring agreement, but this requires repeated decoding and can be sensitive to lexical variation. Semantic self-consistency improves this by clustering sampled answers by meaning using natural language inference, but it adds both sampling cost and external inference overhead. We show that first-token confidence, phi\_first, computed from the normalized entropy of the top-K logits at the first content-bearing answer token of a single greedy decode, matches or modestly exceeds semantic self-consistency on closed-book short-answer factual question answering. Across three 7-8B instruction-tuned models and two benchmarks, phi\_first achieves a mean AUROC of 0.820, compared with 0.793 for semantic agreement and 0.791 for standard surface-form self-consistency. A subsumption test shows that phi\_first is moderately to strongly correlated with semantic agreement, and combining the two signals yields only a small AUROC improvement over phi\_first alone. These results suggest that much of the uncertainty information captured by multi-sample agreement is already available in the model's initial token distribution. We argue that phi\_first should be reported as a default low-cost baseline before invoking sampling-based uncertainty estimation.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Hallucination Detection on PopQA n=1000 (test)](<https://www.sota2.com/research/sota/hallucination-detection-on-popqa-n-1000-test.md>): Hallucination Detection; PopQA n=1000 (test); AUROC: 0.895.

- [Hallucination Detection on TriviaQA n=1000 (test)](<https://www.sota2.com/research/sota/hallucination-detection-on-triviaqa-n-1000-test.md>): Hallucination Detection; TriviaQA n=1000 (test); AUROC: 0.794.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2605.05166>)

[PDF](<https://arxiv.org/pdf/2605.05166.pdf>)
