# Detecting Hallucinations in Graph Retrieval-Augmented Generation via Attention Patterns and Semantic Alignment

Source: [Detecting Hallucinations in Graph Retrieval-Augmented Generation via Attention Patterns and Semantic Alignment](<https://www.sota2.com/research/paper/693a501f6cfd488761a54b4d>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-12-12T05:03:34.150Z

Authors: Shanghao Li, Jinda Han, Yibo Wang, Yuanjie Zhu, Zihe Song, Langzhou He, Kenan Kamel A Alghythee, Philip S. Yu

Published: 2025-12-09T21:52:50.000Z

## Abstract

Graph-based Retrieval-Augmented Generation (GraphRAG) enhances Large Language Models (LLMs) by incorporating external knowledge from linearized subgraphs retrieved from knowledge graphs. However, LLMs struggle to interpret the relational and topological information in these inputs, resulting in hallucinations that are inconsistent with the retrieved knowledge. To analyze how LLMs attend to and retain structured knowledge during generation, we propose two lightweight interpretability metrics: Path Reliance Degree (PRD), which measures over-reliance on shortest-path triples, and Semantic Alignment Score (SAS), which assesses how well the model's internal representations align with the retrieved knowledge. Through empirical analysis on a knowledge-based QA task, we identify failure patterns associated with over-reliance on salient paths and weak semantic grounding, as indicated by high PRD and low SAS scores. We further develop a lightweight post-hoc hallucination detector, Graph Grounding and Alignment (GGA), which outperforms strong semantic and confidence-based baselines across AUC and F1. By grounding hallucination analysis in mechanistic interpretability, our work offers insights into how structural limitations in LLMs contribute to hallucinations, informing the design of more reliable GraphRAG systems in the future.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Hallucination Detection on MetaQA 1hop (Qwen2.5-7B)](<https://www.sota2.com/research/sota/hallucination-detection-on-metaqa-1hop-qwen2-5-7b.md>): Hallucination Detection; MetaQA 1hop (Qwen2.5-7B); AUC: 85.28.

- [Hallucination Detection on MetaQA 1hop (LLaMA2-7B)](<https://www.sota2.com/research/sota/hallucination-detection-on-metaqa-1hop-llama2-7b.md>): Hallucination Detection; MetaQA 1hop (LLaMA2-7B); AUC: 83.41.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2512.09148>)

[PDF](<https://arxiv.org/pdf/2512.09148.pdf>)
