# FACTUM: Mechanistic Detection of Citation Hallucination in Long-Form RAG

Source: [FACTUM: Mechanistic Detection of Citation Hallucination in Long-Form RAG](<https://www.sota2.com/research/paper/69647fda53fdd5c3617445b8>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-01T05:00:04.436Z

Authors: Maxime Dassen, Rebecca Kotula, Kenton Murray, Andrew Yates, Dawn Lawrie, Efsun Kayi, James Mayfield, Kevin Duh

Published: 2026-01-09T15:41:08.000Z

## Abstract

Retrieval-Augmented Generation (RAG) models are critically undermined by citation hallucinations, a deceptive failure where a model cites a source that fails to support its claim. While existing work attributes hallucination to a simple over-reliance on parametric knowledge, we reframe this failure as an evolving, scale-dependent coordination failure between the Attention (reading) and Feed-Forward Network (recalling) pathways. We introduce FACTUM (Framework for Attesting Citation Trustworthiness via Underlying Mechanisms), a framework of four mechanistic scores: Contextual Alignment (CAS), Attention Sink Usage (BAS), Parametric Force (PFS), and Pathway Alignment (PAS). Our analysis reveals that correct citations are consistently marked by higher parametric force (PFS) and greater use of the attention sink (BAS) for information synthesis. Crucially, we find that "one-size-fits-all" theories are insufficient as the signature of correctness evolves with scale: while the 3B model relies on high pathway alignment (PAS), our best-performing 8B detector identifies a shift toward a specialized strategy where pathways provide distinct, orthogonal information. By capturing this complex interplay, FACTUM outperforms state-of-the-art baselines by up to 37.5% in AUC. Our results demonstrate that high parametric force is constructive when successfully coordinated with the Attention pathway, paving the way for more nuanced and reliable RAG systems.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Citation Hallucination Detection on NeuCLIR 2024 (test)](<https://www.sota2.com/research/sota/citation-hallucination-detection-on-neuclir-2024-test.md>): Citation Hallucination Detection; NeuCLIR 2024 (test); AUC: 73.7.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2601.05866>)

[PDF](<https://arxiv.org/pdf/2601.05866.pdf>)